Year Five

The average lifespan for a software engineering job is 4 years. Okay, I've never actually seen proof (or contradiction), but that's the general feeling in the groups I associate with. Perhaps that's selection bias - my employer has generally changed on year 3 or 4. Perhaps this is the exception and not the rule, in that case feel free to simply read this as an experience report. However, I do think it's somewhat common for developers to leave around year 3 or 4. This entry contains speculation on why they leave, and offers one idea on what employers can do to break that cycle.

My 4 year employment cycle generally looks like this

• Year One: "I'm in over my head. My semi-bluff was in-fact a bluff. They're going to fire me any day."
• Year Two: "It's nice to feel like a productive team member"
• Year Three: "This is fun, and I'm not bad at it. It's satisfying to pass on knowledge to teammates."
• Year Four: "This feels repetitive, that grass over there sure looks greener"

I expect that I, like many programmers, probably undervalue my contribution in the early days and overvalue my contribution in the latter days.

In Year Three and Four at DRW I spent some time thinking about how I felt, and observing the behavior of some colleagues that were also on year three and four. A few things stood out to me.

• A company you don't work at always seems to have infinite possibilities; however, after a few years with an employer, it's extremely clear what your options are. More importantly, it's very clear what limitations will likely always be there.
• A company you don't work at contains no code you're responsible for. Conversely, any company you've been with for 4 years probably has plenty of code you're not proud of. If you're responsible for that code, it's a constant reminder of your previous limitations. If you're not responsible for it, your co-workers aren't likely to let you forget about it anytime soon.
• There's always someone willing to pay you more than you're worth. After several years with a company it's likely that they're going to pay you what you're worth, but not what some other company thinks you're worth. I'm surprised that more companies don't pay (the employees they want to keep) what their "flawless market value" would be. In other words, what would you pay them if they interviewed, you determined what they knew, you determined what value they would bring, and you were completely ignorant of their flaws? That's what your competition is likely doing. That's what you're fighting against if you want to keep them around.
• A new job often offers a new challenge. Once you feel like you've given that challenge your best shot, what remains? If you did a great job, it's likely that you'll have plenty of other options. However, if you've done a good job, you may be stuck in a spot where there aren't as many open doors and challenges to choose from - not nearly as many as a position at another company will appear to offer.

I was recently in Punta Cana for wedding, and I was on the beach - working on my laptop. My wife asked: don't you want some time off? My response was short and immediate: no. Later that evening my wife and I discussed my work situation. I observed that I'm in Year Five at DRW and I'm happy, happier than ever, strange - given my previous experiences. She asked if I thought that I was working too much, and if I thought that I would burn out. I remarked: I'd rather have a job that I love, that I don't like to be away from, than a job where I feel like I need a week or two off.

I hear you, nice work if you can get it. I don't have a general recipe for getting there, but I know how I got there.

Back in 2009 I interviewed at DRW. At the time I was working for ThoughtWorks, and my client was Forward. I considered the founder of Forward to be a friend and someone I would gladly work for. I decided it was time to leave ThoughtWorks (after 3.5 years), and I was sure that Forward would be my future home. I remarked to my DRW recruiter "H" (who also happened to be a friend from my ThoughtWorks days) that one of the best things about Forward was knowing that I liked and trusted the man who ran Forward. H said nothing, but made a brilliant move.

In my interview I was grilled, killed even, and then things turned. I met with a guy who asked me a few questions and then told me about the company: the vision, the people, and where I could fit in. He was smart, easy to talk to, and someone I related to. We discussed things casually, it didn't feel like a company pitch in any way at all, it felt like small-talk - something I was very grateful for after the beating I'd taken previously in of the day. After everything concluded I hit the bar with my friends, including H. At that point they revealed to me that the guy I'd met was the partner at the firm that was (among other things) responsible for the firm's technology. I'd also met the CTO, and various other people responsible for technology in the firm. H had shown me that DRW, just like Forward, had what I like to call Awesome All the Way Up.**

Awesome All the Way Up has served me very well at DRW. To this day I remain in fairly common contact with the CTO and several of DRW's partners. About 6 months ago I asked 3 favors. First of all, I asked for enough money to pay someone's salary for 6 months. I identified a project that I wanted to undertake, and I needed help to complete it. Then things got unconventional, I asked if I could create a contract-to-hire situation. Even more unconventional, I pursued a friend and previous colleague who lived in Austin, Texas. DRW rarely uses contractors, and has no other remote employees that I'm aware of. An appropriate amount of questions were asked, but in the end my request was granted.

The experiment is on-going, but I'm very happy with our progress so far. That's all well-and-good, but the support of DRW is the important aspect of the story. I'm confident that their support of my unconventional requests was a major factor in ensuring my happiness in Year Five. We recently hired John Hume, thus declaring success at some level already. However, if things had gone poorly, both parties could have gone their separate ways with little lost and lessons learned. More importantly to me, DRW would have continued to give me confidence that they were willing to take chances to provide me with opportunities and ensure my continued happiness at the firm.

There's a similar discussion around DRW allowing me to use Clojure as my primary development language. I'll spare you the long version. tl; dr: They gave me a reasonable amount of space to try something new, and supported me appropriately as we found more and more success.

Not all of my experiments are green-lighted, and I've also had unsuccessful outcomes. DRW has done a good job of not setting me up to fail; my ideas that have a low probability of succeeding are fleshed out and appropriately shot down. All experiments have risk measures put in place, limited downside, and are reassessed constantly. It's great to have support when things are going well, and it's essential to have support when things don't go as planned.

For me, that's been the secret for keeping me around more than 4 years: An appropriate amount of trust and a willingness to experiment.

A foreign thought also recently came to mind. For the first time in my life I can say that I see myself happy and successful at my current employer in 10 years. This is a question I've asked many people since it occurred to me. To date, +AdeOshineye (http://www.oshineye.com/) is the only person who's responded affirmatively. The results aren't surprising to me, but I do wonder why more employees and employers aren't looking for ways to extend relationships.

Perhaps the secret for keeping me around isn't more broadly applicable; however, simply asking what will keep an individual around is probably the more important message in this entry. It's good to know what will make someone happy now, but it seems like it's equally important to know what will make them happy in the long term. I suspect the answers will be at least a little, if not very different.

The way things currently stand, I'm looking forward to writing about Year Six.

** DRW became my home in the end; however, Forward continues to do well. I suspect Awesome All the Way Up would have ensured happy and gainful employment at either destination. I remain in regular contact with my friends at Forward.

Signs That Your Project Team Might Be Too Large

Amanda Laucher recently queried the twitterverse on the topic of team size.

In my opinion, my last project team was "too large". This blog entry will focus on the things that happened that drove me to that conclusion.

note: I was the only person on the team who believed it was too large. The rest of the team thought we 'might' be too large, and the cost of making a change was greater than the pain of remaining 'potentially' too large. Since I was the minority, it's safe to assume that my opinion is controversial at a minimum.

In many ways my last project was a very, very well funded start-up. We started small, with 3 developers; however, as the scope of our problem grew so went the size of our team. When the team had reached 7 (5 developers in one office and 2 in another), I began to advocate for a team split. By the time we reached 10 total, I was positive that a split was appropriate. As I stated, we were in start-up mode. I wanted to deliver as fast as possible and I felt that several factors were impacting our ability to produce.

Camps. Technology choices originally drove my desire to split the team. The project had speed requirements that forced the use of Java for some portions of our solution; however, other portions of the codebase could have been written using Java, Clojure, or even Ruby. I held the opinion that anything that could be written in Clojure should be written in Clojure. I felt faster writing code in Clojure (vs Java) and I wanted to deliver as fast as I possibly could. Two of my colleagues agreed with my opinion, 2 abstained, and 2 strongly disagreed with me. The two that disagreed with me thought it was easier to write everything in one language, using one toolset. I completely see their point of view, and acknowledge all of their opinions. They are not wrong, we just have incompatible opinions on software.

I need to be clear here, these were good developers. It would have been easy if they were inferior developers - we would have improved their skills or found them a new home. However, that wasn't the case. These were guys that consistently delivered as much, if not more value than I did - they just wanted to get things done is a way that clashed with my approach. The team was more or less split into 2 camps - pro Clojure and pro Java.

Countless hours were spent discussing how to deal with this difference of opinions. Retrospectives were often composed of 2 types of problems: Problems that we all agreed on and solved, and problems that we would never agree on and never solved. The 2nd types of problems appeared at every retrospective and always led to the same interaction.

• someone (usually me) bitched about the lack consistency on the team
• people from my camp agreed that it was an issue, and spoke in generic terms (not wanting to target or alienate anyone).
• people from the opposing camp made their (completely valid) points.
• the room became awkwardly silent; everyone knew we were at an impass
• someone made a joke to ease tensions
• we moved on, no resolution

This situation infuriated me, but, realistically, there was never going to be a solution that made the entire team happy. Everything that was written in Clojure annoyed the pro Java camp, and anything that wasn't performance sensitive and was written in Java annoyed the pro Clojure camp. I naively believed that the pro Java guys would "see the light", and tried everything I could think of to convince them that Clojure was the right choice - and it was for me, but it (likely) never will be for them. In the end, I decided that advocating Clojure was a waste of time and focused my efforts on getting the team split by compatible opinions.

reviewer feedback

On Jun 18, 2012, Patrick Farley wrote:

I think your points about non-compatible opinions on a team, in the saddest case, can occur on a team of two. Do you suppose two distinct camps (as opposed to a bickering mob) become more likely as team size grows?

I do believe it is more likely that camps occur as the team sizes grow. I believe disagreements are healthy; however, camps are counterproductive - due to too much time spent in arbitration. I do agree that you can have camps on a team size two, and it's possible to have no camps on a team of size 10 - which is why I don't think size X will guarantee camps, but that if you have camps then X might be too many.

If your team has camps with incompatible (yet, perfectly valid) opinions then it's a sign that it's possibly too big, or it has members that would be more effectively used if swapped with someone else in the organization that holds compatible opinions. Or, more precisely, if you have camps (where camp is defined as N or more people) with conflicting opinions, and you have a solution that can be split vertically or horizontally, then your team size may be "too big" - where too big is defined as "a split would likely increase the productivity of the members of both camps". It could be possible to replace X split-away members with Y new members where (Y > X) and as long as you do not have camps, the team would not be considered too big - which is why you cannot simply state that Z people on a team is 'too big'. note: N is defined as the number of people required to reasonably address bus risk. On a highly functional team with almost no turnover, N can equal 2. More often, N will be 3 or more.

Depth Shortage. Agile Enthusiasts loved to point out the benefits of Collective Code Ownership - and they are undeniable when contrasted with Strong Code Ownership. However, I believe that Collective Code Ownership actually hindered the delivery pace of the previously referenced development team. Collective Code Ownership combined with our team size resulted in many team members having breadth of knowledge, but lacking depth of knowledge of the system.

reviewer feedback

On Jun 18, 2012, J. B. Rainsberger wrote:

Do you believe that CCO hindered the pace of delivery, or that the code base grew to a size and level of complication that made it hard to own collectively?

Definitely the latter. Our problem had grown to a size that caused CCO to go from a positive impact to a negative one. This was not due to a flaw in CCO, this was due to applying CCO to a situation where it was no longer the most effective choice.

The problem we were solving had 3rd party, distributed system, compliance, extreme performance, and algorithmic concerns that all needed attention. On the surface it didn't seem like a tough problem to solve, and yet, not a single member of the best team I've ever worked on could explain the entire problem. In short, it wasn't just complicated, it was deceptively complicated - which made matters worse. The problem was large, and we attempted to tackle it (clearly, not intentionally) by building a team that was too large.

After a year of practicing Collective Code Ownership I found that most areas of the codebase would have a primary person who deeply understood it's purpose, and almost everyone else knew very, very little about it. Those with shallow knowledge could successfully edit a method, but a change of greater size would require pairing with the primary person. The problem was large, and we were spread too thin.

The application was already split into logical components, and my solution was to move to a Weak Code Ownership model - with the requirement that each component would have at least 3 "owners" that deeply understood each component. You would still be required to know what each component did, but you wouldn't ever be required to even look at the code for that component. I believe this solution would have solved our depth issue, but so would breaking up our large team.

If your team is so large that working in every area of the codebase results in shallow knowledge of over 40% of your codebase, that might be a sign that you could benefit from splitting the team into smaller groups with more specific responsibilities.

And, just in case you are wondering, pairing isn't the solution to having a team that is too big.

I'd like to thank the following people for reviewing and providing feedback on this entry: J. B. Rainsberger, Peter Royal, Nick Drew, Mike Rettig, John Hume, Clinton Begin, Pat Kua, Nat Pryce, Patrick Farley, & Dan North.

The Single Best Thing For My Career

In 2004 I was working at nelnet - for one of the best bosses I've ever had. I was the lone developer working on NextGen projects, our customers were happy, I was paid well, & I had plenty of vacation - things were good. Truthfully, I felt like I was on top of the world. However, I couldn't help but feel like something was missing. I'd seen a bit of 'how things could be' by reading Refactoring & Extreme Programming Explained. I was 24 years old, and I felt like my current situation wasn't going to allow me to grow much. I'm convinced that if I were older I wouldn't have changed a thing, but I was young, and I dove into the deep-end of consulting.

Like I mentioned, I'd seen a bit of the light before I made the jump. I had read a few good books, I was using Test Driven Development (TDD) and Continuous Integration at work, and I'd seen the (successful) results of my efforts. Before I joined the NextGen team I was working with a larger group of developers, and at one point we were in the middle of delivering a huge release. While everyone else was working weekends I was either not in the office or I was at my desk reading about poker theory. My component of the release worked, 100% of the time. My tests prevented regressions even as the release evolved. Through the many month ordeal, I never once had an issue. I had seen the benefits of TDD, there was no turning back, and I wanted to mature my skills even further.

I joined ThoughtWorks in early 2005. From the moment I walked into the office I felt like I was drinking from a fire-hose. What's firefox? Better than IE, okay then. What's DI? Inject the dependencies so you can use mocks or stubs in the tests, got it. Ruby on Rails, PostgreSQL, I'm on it. Then there's all the other things that aren't technology related, but are still worth knowing: international travel, frequent flyer programs, points credit cards, car rentals, etc etc. In my first year with ThoughtWorks I grew more as a person than I had in the previous 3 years combined.

Obviously, it's not all roses. In my orientation class I remember everyone admitting that they were going to put in 2 years for the experience and get out - I had the same plan. The guy with the most experience made it 6 months, two other guys made it a year, and I stayed for 3. The constant travel can wear on you, but it's worth doing it for as long as you possibly can.

There's three reasons that consulting was like steroids for my career:

• diversity of experience
• building my network
• focus on my craft

Each project of my consulting career was very different: C# in the Travel Industry, VB.net in Insurance, Ruby (& DSLs) in Credit Cards, Ruby on Rails (RoR) in VoIP, RoR in Auctions, RoR in Advertising. Projects often used different operating systems (ms, linux, apple) and databases (mysql, postgres, oracle, sql server). Naturally, I couldn't become an expert in everything I touched, but the exposure gave me views into each world. Additionally, I was able to meet experts from each of those worlds. When I joined ThoughtWorks I was a C# developer carrying a Dell, when I walked out I was carrying a MacBook Air and I was best known in the Ruby space - consulting completely changed the trajectory of my career path.

As I mentioned, I often got the opportunity to meet with various experts. I don't know Oracle well, but if I have a problem, I know I can email Pramod. Even though he never worked there, I met Stu Halloway while working for ThoughtWorks. Stu introduced me to Rich Hickey, which is pretty nice, since I spend the majority of my time working with Clojure these days. I met Martin Fowler and eventually wrote a book with him. My network gets me invited to conferences, allows me to co-organize speakerconf, and when I joined DRW, I brought 13 friends with me. If you believe that "your network determines your net-worth", then the majority of my net-worth is due to what I built while I was consulting.

At speakerconf 2011 I noted that everyone involved in the Software Craftsmanship movement was either a consultant or had very recently left consulting. The question of why this was the case was sent to the twitterverse. Brian Guthrie's response rang very true (it was something along the lines of): as a consultant, our craft is the only thing we own. Consultants hone their craft endlessly. I'm convinced that's the reason that consultants are almost always ahead of the technology curve. As a consultant, to invest in your business is to invest in yourself. If you work for a consultancy you'll not only have the time to focus on your craft, you'll also likely be surrounded by a group of people who have been doing the same - and are happy to show you what they've learned.

These days, I'm no longer at ThoughtWorks. In the end, I couldn't deal with the travel or the business model. However, that doesn't change the fact that it was the single best thing that has ever happened to my career. I would highly recommend consulting to everyone who is truly serious about maturing their software development skills.

note: Not all consultancies seem to require as much travel, Relevance, for example, requires very little travel - from what I hear. I also hear that it's an amazing place to work. It would be near the top of my list if I were to get back into consulting.

Random Thoughts on Good Programmers and Good Code

I've know some really, really smart guys who write code I hate. I love using their libraries, but I don't ever want to edit their source. To me, their source is too complicated. I used to think that they were smarter than I was and I benefited by having to write clearer code, as I couldn't easily digest their complicated code.

But, I've also run into code I've written that is complicated in it's own ways. I like to find the dark corners of languages and exploit them. To me, 8 different classes being configured 10 different ways and used in 3 different situations is something I can never get comfortable with. However, metaprogramming & macros don't scare me at all, and I often (over) embrace them.

And, to me, that's the heart of the issue. Each of us have a comfort level with different styles of development. I like to write as little as possible, and embrace "magic" that allows me to write less. Other people are more comfortable with more code, as long as it follows patterns that are easy to follow through the system. We could decide which is better for "average" programmers, but I don't think there's a "better" approach for good programmers who have tried both and stick with what they prefer; they know to play to their strengths.

This also helps explain why all code we haven't written "sucks" and all code we write is "awesome"

As an example, I think test names are worthless because I don't use them. Dan North loves test names and uses them to help him understand the responsibility of code. It's as simple as a different approach for digesting code, but the result is assigning a value to an artifact. I can write all I want about how test names are worthless, but it's not going to change the fact that they are helpful to Dan. He and I could (ignorantly) argue till the end of time about the value of test names, but the actual discussion is in how we digest code.

These days, the Java, Ruby, Clojure, Patterns, Metaprogramming, Static, Dynamic, etc discussions all feel this way to me. You're either an idiot who doesn't understand what you're arguing against, or you're talented enough that you found what makes you the most productive. Either way, your opinions don't necessarily apply to anyone else.

That said, I think there is value in presenting new ideas (or restating old ideas in new ways) as it can inspire people to reevaluate their assumptions. The ideas that tend to be most helpful for me are the ones presented as experience reports and discussions on the benefits of a particular solution. The ideas that are presented as an X vs Y smackdown always tend to make me wonder if someone is still searching for a silver bullet.

Compatible Opinions on Software

I'm switching teams this month. I'll be working out of the DRW New York City office in the near future. I've been with my current team for almost 2 years, and I've learned plenty of valuable lessons. I'm not sure if the most valuable take-away centers around Compatible Opinions on Software; however, it's definitely the take-away that seems to remain in the fore-front of my memories.

The guys I work with in the DRW CT office are truly talented. Each individual on my team is someone I would consider to be a great software developer. As a result, we don't tend to have discussions on issues that where one answer is clearly right and another is clearly wrong. Instead, we often find ourselves in situations where one or more individuals have differing opinions and a "correct" answer cannot be found. Despite everyone being an excellent developer, several members of the team have incompatible opinions on software.

Here's the net result: Quality software. We aren't talking about a problem that caused projects to fail. On the contrary, our customer is very happy and the team is largely happy with the product we deliver. Unfortunately, as a teammate puts it, I seem to be immune to complacency. Even though things are good, I wont be satisfied until they are great. And, I feel that incompatible opinions on software is our largest production thief.

In general, it seems that anything can cause incompatible opinions. However, to make clear what I'm talking about, here is a short sample list of incompatible opinions I've run into:

• powerful language vs. powerful IDE
• monolithic application vs many small components
• what level of duplication is acceptable, and why
• specialization vs generalization
• team size

For the majority of my career I've believed - if I work on a team of excellent developers everything will simply fall into place. I now believe that being technically excellent is only the first requirement. I also believe that you can have two technically excellent people who have vastly different opinions on the most effective way to deliver software.

Two developers who prefer a powerful language may or may not be more productive than two developers who prefer a powerful IDE. However, both of the previous pairs will be more productive than one developer who prefers a powerful language teamed up with one developer who prefers a powerful IDE.

I'm not talking about people who love IntelliJ but are perfectly willing to learn emacs. If you're open to working in emacs then you have a compatible opinion with an emacs zealot. I'm talking about people who have tried the opposing point of view and still have a strong opinion that it's not the best way to deliver software. Putting those people on the same team only ensures a never-ending battle about why things aren't as good as they could be.

Of course, having differing views can be a good thing. Diversity ensures all kinds of benefits. I'm not suggesting that you rid yourself of all diversity. However, for the sake of productivity, I suggest that you do your best to avoid working with someone who has both an opposing view and is as inflexible as you are on the subject. The more central the subject is to the project, the more likely it is that productivity will be lost.

Let's get back to a concrete example. For me personally, I strongly prefer Clojure to Java. That's not to say that Java doesn't have it's place, it does. But, if both languages are valid choices, I'll always opt for Clojure. Given that opinion, I'm more effective if I'm working with someone else that prefers Clojure to Java. The language decision is important to me, very central to the project, and likely to be an ongoing issue.

Building off of the previous example, I don't personally care if we use IntelliJ or Emacs. As long as the IDE or editor choice is a reasonable one, I'm happy to use whatever someone else on the team prefers. I welcome diversity on compatible opinions. In fact, it's likely that diversity on compatible opinions will lead to greater productivity as better solutions are found.

So what can you do? The obvious answers: I've started making a list of opinions that I'm less flexible about. If I'm interviewing for a teammate, I'll ask questions to ensure we have compatible opinions on software. If I'm interviewing for a new job, I'll ask the interviewers about their opinions on software. To a certain extent I've always asked these questions, but I think I'll have better results if I identify which of my opinions I'm less flexible about and focus a bit more on ensuring that there aren't any conflicts.

If you already have a team in place and aren't looking to leave it's a bit harder to resolve incompatible opinions on software. Depending on the size of your team, you may be able to split the team up into smaller groups of people who have compatible opinions on software. Obviously, this will depend on having enough people to allow for a split, and having software that can be logically divided. If you can pull this off, you also get the nice benefit of seeing the results from doing things differently, without having to change your approach until it makes sense for you.

At this point I've worked on three different teams that achieved a level of productivity greater than that of any other teams I've been a part of. Of course, there are several factors that contributed to the success of all three teams. However, all three teams had compatible opinions on software, and avoided an entire class of problems that erode productivity. You'd have a hard time convincing me that those two things weren't somehow related.

Pressure, Expressed in Initial Development Time

def Initial Development Time: In software development projects, initial development time (IDT) is the length of time it takes from the project's first line of code until the business derives notable value from it.

I've done plenty of projects in my career, some with an IDT of a few months and some with an IDT of a year or more. Based on those projects I've decided that I like the following equation to express the pressure felt by a team at any given moment during the IDT.

pressure = Fibonacci(current month of IDT)

note:This equation assumes all other variables are normal. Obviously a team that must finish a project in a month or suffer dire consequences will feel a much different amount of pressure.

During the first month, the team feels 0 (which represents almost no) pressure. There's no legacy code, few broken builds, no requirements issues. During the second and the third month, the team feels 1 (which represents more, but still very little) pressure. The first mistakes and hurdles have been found. The architecture is being pushed.

The following months follow the fibonacci sequence. At month four, the team feels a pressure of 2. At month five, the team feels a pressure of 3. And, so on.

The sequence isn't meant to be out of 100, 1000, or any other number. It's purpose is to show the progression of pressure as the as months of IDT pass. In an ideal world, a manager would understand that a team under extreme pressure will underperform, and the manager would do everything in his or her power to reduce the IDT.

Consequences
Pressure can be good in moderation. Many Agile teams have iteration point goals which pressure the team to get a reasonable amount of work done per iteration. Unfortunately, pressure also tends to have the boiling frog effect.

Pressure often leads to compromises on quality. Far too often I've heard "let's just get this done, and then we'll clean it up." This is a good strategy if you are planning on cleaning immediately after completion (red/green/refactor). However, if you plan on cleaning it up "in the future", that step (the majority of the time) is forgotten.

Fibonacci works well for this (quality related) equation also.

% of code that "sucks" = min(80, Fibonacci(current month of IDT))

Most people wont admit that the code they are currently writing is poor, which helps explain why the percentage rarely reaches over 80%. Also, there's often some code that was written early, and never needed to be modified. Late in a project, looking at that code may be the only thing that keeps you sane.

sidenote: 50% of the time "this code sucks" can be directly translated to "I didn't write this code". On a project with many months of IDT you are bound to experience some turn-over, thus the amount of code that "sucks" is destined to go up. And, even if the original code didn't suck, the modifications done by the person who only half-understands the code probably do suck.

Innovation also seems to suffer on projects with many months of IDT. A typical (well functioning) project team tends to spend 20-30% of their time experimenting with new ideas that will make them more productive in the future. When innovations are found the dividends often greatly exceed the original time investment. Even when no innovations are found, the team often benefits from the experience gained.

Unfortunately, during IDT people begin to forget the value in innovation. They seem to visualize a path to the finish and charge down that path full speed with no regard for hurdles or environment changes. They are often unable to see the value in any time spent that doesn't already have "proven" value. Very soon, you begin to hear: "let's just get this done, and then we'll look at better solutions."

Between (unclean? dirty?) poorly written code and lack of innovation, the chances of dragging a code base out of darkness dwindle quickly as the IDT months drag on.

Lies
Many Agile believers will tell you that the problems with pressure can be alleviated with Milestones. This is patently false. A solution isn't valuable to a business until it provides notable business value. You may be 13 iterations into the project with 2 milestones behind you, but if the business isn't deriving value, you haven't done anything more than show them partially working software. Partially working software is great for building trust, but it alleviates zero pressure with respect to IDT.

Options
The only real remedy to the negative consequences of a long IDT is to create a shorter IDT. There truly is no alternative.

Unfortunately, some projects truly do take many months to deliver business value. I've found that some pressure can be alleviated if the team recognizes and adjusts to the pressure.

The 3 most helpful things I've found are:

• Weekly (scheduled) 1 on 1 discussions between the team lead and each member of the team.
• Take vacation. One person out for a week isn't nearly as catastrophic as a team full of burnt out people.
• Set aside, and always use, experimentation time. Innovations can give you a big boost that the project will definitely need in the long run.

Thanks to Lance Walton for feedback, and the phrase "Initial Development Time"

Staying Current: A Software Developer's Responsibility

I have a personal hatred for weekend conferences*. To me, a weekend conference ensures that I'll be "working" for 12 straight days.

I understand that opinion isn't universal.

Some people have problems getting time "off" to attend conferences. These situations feel like a fundamental misunderstanding of a software developer's responsibilities. Part of your (software developing) job is staying up on current technologies. That means doing some research during your day.

(almost directly stolen from Ward on technical debt)
If you spend your entire day coding and never looking at new things, you accrue productivity debt. In the short term (say the last week of a release), it makes sense to take on a little debt. However, in the long term, assuming little or no payment, the interest (where interest equals the gap between your skills and the current solutions) will render you a NZPP (Net-Zero Producing Programmer). In a typical organization you can coast as a NZPP for around 6 months and slowly transition to a NNPP.

It is your responsibility not to become a NZPP (or NNPP). Most talented software developers refuse to work with NZPPs. At the point that you become NZPP, you usually have to declare bankruptcy (with regard to software development). You generally have two choices: take a much lower paying job where you can learn new things or move into another role. If you want to be a software developer, neither of these outcomes is desirable.

Luckily, you have the power to avoid becoming a NZPP. Most employers will happily buy you technical books and send you to technical conferences. In my opinion, whether or not you took advantage of these benefits should be noted on your performance review. Not staying current as a software developer, especially when the opportunity is offered to you, is software malpractice.

I once created a list of things I look for in potential team-mates.

• Have you tried Test Driven Development? Can you name something you liked and something you disliked?
• What language(s) that are gaining popularity, but not yet mainstream, have you written Hello World in?
• Do you read books or blogs looking for new ideas at least (on average) once every two weeks?
• Do you at least attempt to learn a new language every year?
• Have you ever run a code coverage or cyclomatic complexity tool against your codebase?

A commenter said something along the lines of

Not everyone has the personal free time to dedicate to doing all of these things

And, that is the fundamental flaw. Employees (and even some employers) seem to think that these are activities that should be done in your off time. I can't disagree more. These are things that a responsible developer needs to do as part of their job, thus it can be done during work hours.

20% time isn't something Google invented, it's just something they named, formalized and made popular. The activity itself is something good software developers have been doing for years. I applaud Google for making it a standard, thus ensuring that it's employees always have the opportunity to stay current. However, your company doesn't need to standardize on 20% time for you to stay current.

It's your responsibility to make time in your day to read a book or a blog.

You should also take advantage of a company sponsored trip to a conference. If you've attended conferences before and derived little value, I highly suggest the QCon conferences and JAOO.

Once you start doing research as part of your job you'll find that conferences are just like work, except the focus is 100% on research. And, it's not something you want (or should have to) spend your personal time on, it's just another productive day of doing what you have a responsibility to do.


* Which is why Josh and I run SpeakerConf Tuesday-Thursday. You can travel to, attend and travel home without missing a weekend day.

Thoughts on Developer Testing

This morning I read Joel: From Podcast 38 and it reminded me how immature developers are when it comes to testing. In the entry Joel says:

a lot of people write to me, after reading The Joel Test, to say, "You should have a 13th thing on here: Unit Testing, 100% unit tests of all your code."

At that point my interest is already piqued. Unit Testing 100% of your code is a terrible goal and I'm wondering where Joel is going to go with the entry. Overall I like the the entry (which is really a transcribed discussion), but two things in the entry left me feeling uneasy.

• Joel doesn't come out and say it, but I got the impression he's ready to throw the baby out with the bath water. Unit testing 100% of your code is a terrible goal, but that doesn't mean unit testing is a bad idea. Unit testing is very helpful, when done in a way that provides a positive return on investment (ROI).
• Jeff hits it dead on when he says:

  ...what matters is what you deliver to the customer...

  Unfortunately, I think he's missing one reality: Often, teams don't know what will make them more effective at delivering.

I think the underlying problem is: People don't know why they are doing the things they do.

A Painful Path
Say you read Unit Testing Tips: Write Maintainable Unit Tests That Will Save You Time And Tears and decide that Roy has shown you the light. You're going to write all your tests with Roy's suggestions in mind. You get the entire team to read Roy's article and everyone adopts the patterns.

All's well until you start accidently breaking tests that someone else wrote and you can't figure out why. It turns out that some object created in the setup method is causing unexpected failures after your 'minor' change created an unexpected side-effect. So, now you've been burned by setup and you remember the blog entry by Jim Newkirk where he discussed Why you should not use SetUp and TearDown in NUnit. Shit.

You do more research on setup and stumble upon Inline Setup. You can entirely relate and go on a mission to switch all the tests to xUnit.net, since xUnit.net removes the concept of setup entirely.

Everything looks good initially, but then a few constructors start needing more dependencies. Every test creates it's own instance of an object; you moved the object creation out of the setup and into each individual test. So now every test that creates that object needs to be updated. It becomes painful every time you add an argument to a constructor. Shit. Again.

The Source of Your Pain
The problem is, you never asked yourself why. Why are you writing tests in the first place? Each testing practice you've chosen, what value is it providing you?

Your intentions were good. You want to write better software, so you followed some reasonable advice. But, now your life sucks. Your tests aren't providing a positive ROI, and if you keep going down this path you'll inevitably conclude that testing is stupid and it should be abandoned.

Industry Experts
Unfortunately, you can't write better software by blindly following dogma of 'industry experts'.

First of all, I'm not even sure we have any industry experts on developer testing. Rarely do I find consistently valuable advice about testing. Relevance, who employs some of the best developers in the world, used to put 100% code coverage in their contracts. Today, that's gone, and you can find Stu discussing How To Fail With 100% Code Coverage. ObjectMother, which was once praised as brilliant, has now been widely replaced by Test Data Builders. I've definitely written my fair share of stupid ideas. And, the examples go on and on.

We're still figuring this stuff out. All of us.

Enlightenment
There may not be experts on developer testing, but there are good ideas around specific contexts. Recognizing that there are smart people with contextually valuable ideas about testing is very liberating. Suddenly you don't need to look for the testing silver-bullet, instead you have various patterns available (some conflicting) that may or may not provide you value based on your working context.

Life would be a lot easier if someone could direct you to the patterns that will work best for you, unfortunately we're not at that level of maturity. It's true that if you pick patterns that don't work well for your context, you definitely wont see positive ROI from testing in the short term. But, you will have gained experience that you can use in the future to be more effective.

It's helpful to remember that there aren't testing silver-bullets, that way you wont get lead down the wrong path when you see someone recommending 100% code coverage or other drastic and often dogmatic approaches to developer testing.

Today's Landscape
Today's testing patterns are like beta software. The patterns have been tested internally, but are rarely proven in the wild. As such, the patterns will sometimes work given the right context, and other times they will shit the bed.

I focus pretty heavily on testing and I've definitely seen my fair-share of test pain. I once joined a team that spent 75% of their time writing tests and 25% of their time delivering features. Not a member of the team was happy with the situation, but the business demanded massively unmaintainable Fit tests.

Of course, we didn't start out spending 75% of our time writing Fit tests. As the project grew in size, so did the effort needed to maintain the Fit tests. That kind of problem creeps up on a team. You start by spending 30% of your time writing tests, but before you know it, the tests are an unmaintainable mess. This is where I think Jeff's comments, with regard to writing tests that enable delivery, fall a bit short. Early on, Fit provided positive ROI. However, eventually, Fit's ROI turned negative. Unfortunately, by then the business demanded a Fit test for every feature delivered. We dug ourselves a hole we couldn't get out of.

The problem wasn't the tool. It was how the process relied on the Fit tests. The developers were required to write and maintain their functional tests using Fit, simply because Fit provided a pretty, business readable output. We should have simply created a nice looking output for our NUnit tests instead. Using Fit hurt, because we were doing it wrong.

The current lack of maturity around developer testing makes it hard to make the right choice when picking testing tools and practices. However, the only way to improve is to keep innovating and maturing the current solutions.

If It Hurts, You're Doing It Wrong
Doing it right is hard. The first step is understanding why you use the patterns you've chosen. I've written before about the importance of context. I can explain, in detail, my reasons for every pattern I use while testing. I've found that having motivating factors for each testing pattern choice is critical for ensuring that testing doesn't hurt.

Being pragmatic about testing patterns also helps. Sometimes your favorite testing pattern wont fit your current project. You'll have to let it go and move on. For example, on my current Java project each test method has a descriptive name. I maintain that (like methods and classes) some tests are descriptive enough that a name is superfluous, but since JUnit doesn't allow me to create anonymous test methods I take the path of least resistance. I could write my own Java testing framework and convince the team to use it, but it would probably hurt. The most productive way to test Java applications is with JUnit, and if I did anything else, I'd be doing it wrong.

I can think of countless examples of people doing it wrong and dismissing the value of a contextually effective testing pattern. The biggest example is fragile mocking. If your mocks are constantly, unexpectedly failing, you're doing something wrong. It's likely that your tests suffer from High Implementation Specification. Your tests might be improved by replacing some mocks with stubs. Or, it's possible that your domain model could be written in a superior way that allowed more state based testing. There's no single right answer, because your context determines the best choice.

Another common pain point in testing is duplicate code. People go to great lengths to remove duplication, often at the expense of readability. Setup methods, contexts, and helper methods are all band-aids for larger problems. The result of these band-aids is tests that are painful to maintain. However, there are other options. In the sensationally named entry Duplicate Code in Your Tests I list 3 techniques that I've found to be vastly superior to setup, contexts and helper methods. If those techniques work for you, that's great. If they don't, don't just shove your trash in setup and call it a day. Look for your own testing innovations that the rest of us may benefit from.

If something hurts, don't look for a solution that hurts slightly less, find something that is a joy to work with. And, share it with the rest of us.

Tests Should Make You More Effective
What characterizes something as 'effective' can vary widely based on your context.

Some software must be correct or people die. This software obviously requires thorough testing. Other software systems are large and need to evolve at a fairly rapid pace. Delivering at a rapid pace while adding features almost always requires a fairly comprehensive test suite, to ensure that regression bugs don't slip in.

Conversely, some software is internal and not mission critical. In that case, unhandled exceptions aren't really a big deal and testing is clearly not as high a priority. Other systems are small and rewritten on a fairly consistent basis, thus spending time on thorough testing is likely a waste. If a system is small, short lived, or less-important, a few high level tests are probably all you'll really need.

All of the example environments and each other type of environment share one common trait: You should always look at your context and see what kind of tests and what level of testing will make you more effective.

Tests Are Tools
The tests are really nothing more than a means to an end. You don't need tests for the sake of having tests, you need malleable software, bullet-proof software, internal software, or some other type of software. Testing is simply another tool that you can use to decrease the amount of time it takes to get your job done.

Testing can help you-

• Design
• Protect against regression
• Achieve sign-off
• Increase customer interaction
• Document the system
• Refactor confidently
• Ensure the system works correctly
• ...

Conclusion
When asking how and what you should test, start by thinking about what the goal of your project is. Once you understand your goal, select the tests that will help you achieve your goal. Different goals will definitely warrant using different testing patterns. If you start using a specific testing pattern and it hurts, you're probably using a pattern you don't need, or you've implemented the pattern incorrectly. Remember, we're all still figuring this out, so there's not really patterns that are right; just patterns that are right in a given context.

[Thanks to Jack Bolles, Nat Pryce, Mike Mason, Dan Bodart, Carlos Villela, Martin Fowler, and Darren Hobbs for feedback on this entry]

The Cost of Net Negative Producing Programmers

It's common to compare ourselves to doctors, lawyers, and other highly-paid professionals. Unfortunately, almost every comparison breaks down dramatically, very quickly.

When doctors screw up (massively) they get sued. When (permanently employed) programmers screw up they get let-go, and they go find a new job, usually with more responsibility and a pay raise. There are no ramifications for software malpractice.*

Successful lawyers put in years to learn their craft, then they put in years trying to become partner. Eventually they get to make the firm defining decisions, but only after maturing their abilities and proving themselves. In our industry you need no formal education. If you show the slightest sign of competency you'll quickly be given the keys to the kingdom. Architect promotions are not hard to come by.

I had an 'architect' title with no college degree and only 3 years of experience. At that point I'd never heard of unit testing, refactoring, or design patterns. In those days I was picking up information from O'Reilly In a Nutshell books and MSDN. At the same time I was leading a team building software for the US government (via contracts the company had won, not employed directly by them). I was massively under-skilled, and yet, there I was writing software that troops would need to stay alive.**

I wish my experience were isolated, but while I was consulting for 3.5 years I saw countless examples of exactly the same story.

I know the argument: demand is so high, we don't have another choice. I reject this argument on the basis that most good programmers spend the majority of their time fixing problems created by terrible programmers.

Good programmers fix problems created by terrible programmers in various ways. Good programmers can directly address problems by rewriting poorly written custom code. However, the less obvious way good programmers address poorly written code is when they write custom code because all the tools, that a company could potentially buy instead of building, are terrible. Would so many companies write their own time tracking, project managing, bug tracking, expense tracking and other internal tools if they had reasonable commercial options?

The argument that demand is too high completely ignores the opportunity cost of working with NNPPs.

The next common argument: We don't have that many NNPPs. Really? Write a few blog entries, attend a few conferences, do some consulting. I think you might change your mind. And, remember, the people commenting on blog entries or attending conferences are the ones who actually care about what they are doing. I'm horrified to think what the less interested NNPPs produce.

Here's a gem from a comment on my last blog post.

I honestly think I can do about 3000 lines of good code in a day... -anonymous

The commenter actually thinks writing 3000 lines of code a day is a good thing.

If you read through the comments you'll find another commenter that doesn't understand the difference between open classes and inheritance, but overall the majority of comments are well thought out reasonable responses. Several people were able to create logical responses without being emotionally attached to Java. That gave me some hope that things were a bit better than I generally picture them as. But, then I checked out the dzone score of the entry.

Now, maybe it's just not a well written or educational in any way post, that would be fine. But, when I read the comments, I'm back to disgusted by our industry. In this day and age, is it really reasonable to think that Java doesn't have limitations? I would say no. Java is a great tool for certain tasks, but there are plenty of things to dislike about it. I wouldn't want to work with people too blind to notice that.

Another common statement is: In every industry you have people who don't care about their jobs. I don't think that's a good comparison either. Bad doctors are sued until they can't afford malpractice insurance. Lawyers, very publicly, lose cases and are fired. In those industries, if you aren't contributing towards moving things forward, you're quickly exited.

Professional sports is an industry that probably has very few professionals that don't care about their job. If you aren't good enough, you don't make it, period. In basketball, there's no position for someone who is only good enough to dribble. If you're good enough, you're paid well. If you aren't, you don't make it. It's that simple.

So what is the cost of NNPPs? I'd say there are several ways to answer that question.

The first obvious cost is opportunity cost that companies lose when they can't provide quality software to their customers. This can translate to significantly lower revenue due to lack of or cancelled subscriptions or licenses. It can be the difference between the success and the failure of a start-up. For businesses, I would say the cost is epic. Is it any surprise that technology companies are some of the most volatile stocks.

There's other costs as well. When software fails, people don't get the aid they needed. This can be money, hospital care, legal direction, and many other life altering situations. Poorly designed software can cause death, and yet rarely is that kind of thing considered by a programmer.

I heard once that in Great Britain's MOD if you design software for a plane you go up in the test plane when the software is beta-tested. If all programmers were held with that level of accountability, how many do you think would still be in our field? How many would you want to collaborate with before you went up in the plane together?

Of course, we don't all write life-threatening software, but does that give us an excuse for lowering our colleague-quality requirements? Picture the things we could do if we didn't spend most of our time making up for terrible programmers and you'll know why I'm passionate about the topic.

Terrible programmers also slow us down as an industry. When I talk about Open Classes people are terrified. They always say: That's fine on your team, but we could never work with a language that has Open Classes. Is that a problem with Open Classes or a problem with the team? I worked on several teams, large and small, that used Open Classes diligently and I can't remember a single problem we ever had with them. Much the opposite, the teams were often, clearly more effective because they were talented and the language let them solve problems in the most effective way.

Java and C# are not silver bullets. The languages are good solutions to a certain class of problems, using them for problems that could be better solved with a different language stagnates the growth of other languages. The longer great programmers use a less-effective tool for the job the less time they have to work with and mature languages that are more suitable. As a result our entire industry loses out.

There's also a cost to your future colleagues. There's a big difference between a NNPP and someone new to the industry. Someone new to the industry benefits greatly from a mentor, but what if the mentor is an NNPP? Some NNPPs do small scale damage in isolated components of applications. But, the most insidious NNPPs are the architects who's ideas belong on The Daily WTF. These NNPP architects build entire armies of NNPPs around them and single-handedly waste millions, if not billions of dollars, annually. And, potentially worse, they ruin or at least drastically hurt the careers of eager to learn teammates.

The Cost of NNPPs is high enough that it's become my soapbox issue. But, truthfully, I'm not saying anything really new, so is there any hope for our industry?

I do think there are things we can do to help move the industry in the right direction. The good programmers can refuse to work with bad programmers. That might mean moving to an organization where that's a goal, or making that a goal of your current organization. Providing negative feedback directly to a NNPP teammate is always hard, but I do believe the ROI justifies the action. It's also helpful to provide that feedback to your manager, so the manager knows your opinions on the skill levels of your teammates. You can also suggest to managers that employees who refuse to take advantage of training budgets should be looked closely at. Lastly, you could suggest moving developer compensation to be based on the success of the project. A royalties model would be really interesting.

And, if you have a blog, you could write your own entry expressing your opinions on the topic.

* The exception being freelance developers, who are the minority, at least here in the US.
** Thank God, the government never ended up using the software we delivered.

Targeted Languages

The vast majority of actively evolving business software is written in Java these days. Java has long enjoyed the title of One Language to Rule Them All. However, in a previous post, The Next Big Language, I mention that I'm skeptical that there will be one language that is perfect for solving all possible problems in the future.

I might be overestimating the speed at which our profession is maturing. One of the reasons Java became the enterprise standard was because the wrong people were making decisions based on inadequate information and swarms of terrible programmers. I like to believe that we're moving away from those days, but then again, we definitely still have far too many NNPPs that need to be encouraged to find other employment. However, I do think it's inevitable that we move to languages targeted at specific domains and problems eventually.

We've already taken the first step. Games are a form of business software. Games are consumer products. However, game development has never been dominated by Java. I won't pretend to know about the game industry, but from what I hear it's been largely dominated by C++. However, these days Lua has enjoyed great popularity in the videogames industry.

The game industry has been using the right tool for the job for some time; however, other business are also starting to catch on. Many companies are using Ruby and Rails to build websites. There's no question that having a good group of programmers building your website using Ruby and Rails can provide a significant competitive advantage. Erlang is another language that targets a specific class of applications, and it provides huge advantages over the alternatives for those applications.

I think targeted languages raise some new interesting questions.

For the professional software developer, targeted languages may make it harder to switch domains. These days it's easy to get a job doing Java development in insurance, banking, advertising, and many other domains. If in the future all the banks are using a functional language focused on low-latency, you may find it harder to make the switch to an advertising agency with a shop that's using an object oriented dynamic language. It may not be long before selecting a language implies selecting a domain as well.

There will be implications for firms also. The pools from which companies hire from will get much smaller. It's very hard to hire a good Java developer these days, imagine what the picture is going to look like when there are 5-7 targeted languages that are widely accepted and the developer pool is split 5-7 ways.

There will also be questions about how many languages your organization will support. If you're a bank and you write your trading applications using the best language for the job, would you allow the intranet dev team to use a completely different language?

Companies are already facing these questions. At DRW Trading we use C#, Java, C++, Python, Ruby, Perl, etc. I hear that Google only allows C++, Java, Python, and Javascript. There's definitely a balance between leveraging existing knowledge and using the right tool for the job. However, finding that balance isn't an easy task.

Like it or not, we're headed towards more targeted languages. It's probably worth considering the implications sooner rather than later.

Specialize in Something Relevant

generalist: a person competent in several different fields or activities

If you read my blog entry on Language Specialization you might have concluded that I prefer generalists. If, in our industry, generalists were what the definition describes, then I would prefer generalists. Unfortunately, business software developers seem to have created their own definition of generalist.

business software developing generalist: I know how to do the simplest tasks with many different languages/tools, but I can not be considered competent with any of them.

I blame Scott Ambler. To me anyway, it seems like the daft generalist movement started when Scott wrote Generalizing Specialists.

Our industry has always been saturated by bad programmers. I'm on record stating that at least 50% of the people writing business software should find a new profession. The problem with bad developers is that they take good ideas and turn them in to monstrosities.

I remember reading Generalizing Specialists and being inspired. I thought Scott gave fantastic and relevant advice. Unfortunately, many bad or junior developers heard: Don't bother to deeply understand anything, instead, you're agile if you know a little about everything. Suddenly, when I started interviewing developers I ran into situations like this.

• me: So, I see you have Erlang on your resume, how do you like the language?
• candidate: I like it's concurrency handling, but I'm a bit weary of it's syntax.
• me: (thinking - okay, do you have any original thoughts on Erlang?) I can understand those points of view, what problem were you trying to solve with Erlang and why did you think it was the right tool?
• candidate: Oh, I really only got through the 2 minute tutorial, you know, hello world basically. But if you guys have Erlang projects you want me to work on I'm happy to, I'm a generalist, I like all languages.
• me: Okay, so what language would you say you know the most about?
• candidate: I don't bother to specialize, I do a little bit with each language, you know, hello world or whatever, so I can use the right tool for the job. That's the best part of being a generalist.
• me: (thinking - this interview is already over) Okay, so tell me about the languages/tools you've had to use at your different jobs?

Inevitably, the candidate doesn't even have a deep understanding of the tools they've used at work, because they are too busy doing hello world in every language invented. They also love to say that they take the Pragmatic Programmers advice to extreme and 'learn' several languages a year.

The truth is, these generalists have little in the way of valuable knowledge. They provide their projects with little more knowledge than a Google search can bestow in 30 minutes. In short, they're worthless, if not destructive.

I don't actually blame Scott Ambler. In my opinion he was right then, and he's right now. Become a Generalizing Specialist is still the advice that I currently give developers.

Specializing in something makes you an asset to the team. If I'm building a Web 2.0 website, I want everyone to have an understanding of HTML, CSS, Javascript, Ruby, & SQL. However, I also want each team member to specialize in one of those areas. Knowing IE quirks is just as important as knowing how to optimize MySQL. And, I want to make sure I have team members that can get into the deep, dark corners of delivering highly effective software. That doesn't mean everyone needs to know what a straight join in MySQL does, but at least 1 person should. The rest of the team isn't entirely off the hook though, they better understand how to write basic SQL statements that are maintainable and at least semi-performant.

Becoming a Generalizing Specialist takes time, but the first step is becoming a Specialist. Once you deeply understand one language/tool, you can move on to the next relevant language/tool. How do you know when it's time to move on? When you start having answers to questions that people aren't asking. If you're constantly looking up answers to common questions, you aren't a specialist. However, if you start providing more (relevant) detail in your answers than people are looking for, you're on your way to possessing the deep understanding that a Specialist should have. At that point, it's probably time to start looking deeply into something else.

One painful mistake to look out for is specializing in something less relevant. If you work for a trading firm that writes only thick client applications, understanding why Chrome's Javascript VM is better than Firefox's Javascript VM is probably not the best use of your time. It's true that you may move on to a web application at some point, but by then your information will probably have become stale anyway. Stick to specializing in things that you work with day to day. Your language, your IDE, the Domain Specific Languages you use in your applications (regular expressions, SQL, LINQ, etc), or the frameworks you use (Spring, ASP.net, etc) are things you should specialize in to increase the value you provide to your team.

Eventually, you become competent with several different tools and languages. You've become a Generalizing Specialist and as such you are significantly more valuable to your team.

Language Specialization

Didn't you just totally sell out? -- Obie Fernandez @ Rails Summit Latin America

Obie and I are good friends. He wasn't trying to insult me. I was talking about how much I liked my new job (at DRW Trading), and the different aspects of the job. One aspect of my job is that I spend a fair amount of my time working with Java. I do some C# and some Ruby also, but these days it's more Java than anything else. I believe Obie was genuinely curious if I felt like I sold out since I'm not doing Ruby full-time anymore.

It's an interesting question, but it comes packed with all kinds of assumptions. For the question to be valid, I would have had to trade something I truly care about for the combination of something I did and did not like. Luckily, that wasn't the case.

Obie isn't the first person to be surprised that I'm no longer working full-time with Ruby. Truthfully, I find it a bit funny that people think I would base a career move on a language. Ruby is my favorite language, but it's not the correct choice for every problem that needs to be solved. And, languages have never been my primary concern when deciding what job to take.

My first job primarily used Cold Fusion. When I joined AOL Time Warner I gave up Cold Fusion for PHP. When I joined IAG I gave up PHP for C#. And so on. As you can see, I've never been too tied to a language. I've always been most interested in learning and growing. I love jobs that help me improve my skills.

Chad Fowler talks about something similar. In the section "Don’t Put All Your Eggs in Someone Else’s Basket" of My Job Went to India, Chad says the following:

While managing an application development group, I once asked one of my employees, “What do you want to do with your career? What do you want to be?” I was terribly disppointed by his answer: “I want to be a J2EE architect.” ...

This guy wanted to build his career around a specific technology created by a specific company of which he was not an employee. What if the company goes out of business? What if it let its now-sexy technology become obsolete? Why would you want to trust a technology company with your career?

I think Chad got it right, but it's not just companies you shouldn't trust. I wouldn't base my career on any technology, whether it was produced by a company or an open source community.

I prefer jobs that allow me to learn new things. Think of it as job security -- I shouldn't ever be out-of-date when it comes to technology experience. Think of it as an investment -- everything I learn creates a broader range of experience that I can leverage for future projects or jobs. Think of it as experimenting -- by trying many different solutions I may find ways to combine them and innovate.

I've turned down several jobs that paid more or offered comfort. I've never regretted it. Your career is long and (as Chad says) you should treat it as a business. When you look at it from that perspective it obviously makes sense to spend the early years learning and deciding which is the best direction to take.

The truth is, if you focus on one technology you'll never be as good as your teammates who have more experience mixing technologies to produce the best solution.

As I told Obie, I definitely don't feel like I sold out. In fact, one of the reasons I joined DRW was because they use Java. I've never worked with Java, messaging, or the financial domain. Having experience with those 3 things will make me better. And, diversifying your experience will make you better as well.

When To Retire Your Brand

Building a brand takes a lot of effort, but I think the payoff justifies the investment. Having a strong brand definitely helped me find a fun and very well paying job. So now that I have a dream job (@ DRW Trading), what should I do with my brand?

I have to confess, I didn't start writing because I wanted share information. I started writing because I wanted to build a big brand, find a great job, and enjoy life. Somewhere along the way I began to enjoy writing and the positive results that came from knowledge sharing. Someone once said to me "I write better tests because of your blog". Obviously I was happy to hear that kind of feedback. At the same time it wasn't the reason I got started down this path.

I love to program. I also love to be in shape (which I'm not) and learn other languages (which I haven't been doing lately).

Now that I have my dream job, I've been spending more time doing the other things I love. Unfortunately, I found that between learning an interesting domain, going to the gym, and learning Italian, there's not much time left for blogging. I also find that since I'm doing all the things I love, I don't really like to be away presenting at conferences.

I thought it might be time to declare success on the brand building project, and move on to new pursuits... But, I was wrong.

The largest reason I can't quit writing and presenting is that I enjoy giving back to the community. Seeing a blog entry get 8,000 hits in one day causes an amazing feeling. Giving a presentation and getting feedback that says "Probably the best presentation at the conference" definitely makes you feel good about what you are doing. Seeing an idea become committed as the way to do something will definitely make you smile. I truly enjoy spreading ideas (or at least attempting to spread ideas) that help the community evolve.

Blogging and presenting also help me personally improve. The easiest way to get feedback on something is to put it out there. I considered several of my testing ideas to be "the right way" for far too long. Putting them down as blog entries resulted in further evolution of the ideas as well as a greater understanding of how context determines the correct approach. Simply writing about my ideas improves them. One thing we aren't short of is people to tell you you're wrong.

Your brand is also valuable to your employer. Employing people with name recognition improves your organization's ability to recruit talented new hires. This also directly benefits you, since you'll be given the opportunity to work with more talented teammates. At the moment, DRW is looking to hire the absolute best people in the industry. I wish I had an even stronger brand, so I could help attract the top talent.

Ultimately I came to the conclusion that building a brand is a career long activity. You can stop at any time, but getting that free time back comes at cost to your profession.

Is Distributed Development Viable

I've never seen distributed development succeed. However, before we get into what I've seen, I need to be specific about what I'm describing.

Distributed Development: A group of individuals who work across time, space, and organizational boundaries with links strengthened by webs of communication technology

The link above for Distributed Development redirects to Virtual Team. Wikipedia also has an entry for Distributed Development; however, I found the description of Virtual Team to be much closer to what our industry generally labels Distributed Development.

Just to be clear I'm not talking about outsourcing (subcontracting a process, such as product design or manufacturing, to a third-party company) or what you might call off-shoring.

This entry will be written with that in mind.

Take 1
Several years ago I worked with one of the nicest and most talented developers I expect I'll ever encounter: Badri. We worked together for about a month before he was forced back to India to deal with US visa issues. We knew this was going to happen and decided that it was so important to keep Badri on the project that we were going to give distributed development a shot. Badri knew the code, the team, and he was great on so many levels that the choice couldn't have been more obvious.

After just a few weeks, we gave up on distributed development. Ever since that experience, I've been highly skeptical of the feasibility of distributed development. I had actual experience working with one of the greatest developers I'd ever encountered, and we still failed.

Failure can be good. There are always lessons to be learned when you fail. Here were a few that I picked up from that project.

• Time difference is probably the single largest contributing factor to failure. Badri was in India, and we were in Deleware, USA. There was almost no time overlap to our workdays.
• Having the local or on-site developers feed requirements back to the off-site team equates to an order of magnitude efficiency loss.
• Having the on-site developers decide what pieces are going off-site quickly translates to the off-site team being pissed off about getting the boring work
• Businesses expect the same level of quality and productivity from the on-site members as the off-site members.

Given those circumstances, we definitely failed. Epic failed in fact. And, to make things worse, I was the one that had to tell Badri that we were going to give up on distributed development. Imaging having to tell someone you have nothing but admiration for that "it's just not working out". I still feel guilty about it to this day.

The time difference was a huge killer. Every time we spoke to Badri either he was half asleep or we were. I think Badri was working 12 hour days also, just so he could try to talk to us at the beginning and end of his day. It also meant no wine with dinner or having a bit of a buzz for the 11pm stand-up. Either of those options is unacceptable. Worse, if we didn't communicate what we needed in detail, the off-site team would be off in the wrong direction for up to 12 hours straight. We threw away several days worth of code because we hadn't given them enough detail to go in the right direction.

Truthfully, we (the on-site developers) shouldn't have been deciding what they were going to work on in the first place. It was a tough project and we were all learning. None of us felt like we had the time to do both the development required of us, plus the analysis required to give the off-site team good direction. Instead, the off-site team got minimal direction and delivered code that was of minimal use to us. It was beautiful code, but it wasn't what we needed. Again, epic fail.

Even though we clearly were not functioning well, the business expected that the off-site team deliver at the same pace as the on-site team. The business had met and loved Badri. No one could understand why our pace had suddenly dropped off. Not good. More on this later.

Take 2
A few years later I joined a project that was just beginning to give distributed development a shot. I hadn't been there 3 days and I was already in the CEO's office explaining why I thought it was a bad plan. Of course, I was gun-shy, so that was to be expected. Luckily, Fred George was also there and he was more optimistic. In the end we went to an on-site team, but Fred showed me that distributed development is possible.... maybe.... eventually.

We experienced several obstacles, and we overcame some. Again, there were plenty of lessons to be learned.

Time difference was less of a factor. This time I was working in London and the off-site team was in India. Our workdays overlapped fairly well. This was obviously a huge move in the right direction. We also collaborated on who would work on what, which helped ensure that everyone was happy with what they were working on. And, the analysis came from a BA that collaborated with both portions of the team. The BA even traveled to India occasionally.

However, things were still quite broken.

Some members of the team had never met other members of the team. There's something about working, collaborating with someone in person that is almost impossible to replicate over a phone or IM conversation. It wasn't that we didn't want to get everyone together, but there were visa issues that couldn't be overcome. It was a big mistake with no good resolution. Some long time team members couldn't travel, but if we replaced them with people who could travel we would lose domain knowledge and context.

Communication was a constant problem. The telephones on both ends presented problems. I have plenty of Indian friends that I have no problem understanding, but with the off-site team speaking into a bad connection and it being sent out of our bad connection, I caught every other sentence, if that. To make matters worse, people weren't talking nearly as often as they needed to be. We tried to address this by creating a chat room and mandating daily checkpoint phone calls. Both ideas were abandoned due to lack of buy-in. In the end we never did solve the communication issue -- in my opinion. I think the reality is that most programmers are introverted, and being off-site just gives you one more excuse for not talking to the business, even when you really should.

Connectivity was also an issue. If you're in the US, you don't really even think about the reliability of your internet connection. However, in many other places in the world connectivity is much less certain. There were several occasions where the off-site team was simply unable to check-in their code. We never did figure out if the problem was on their side, our side, in our vpn, or some other bizarre location. If your off-site can't check in, IM you, or get to the wiki, you obviously lose productivity.

There was one common problem: the business expected the same level of productivity from both teams. I'm not sure there's a way around this issue. Have you ever heard that programming is about people? Of course you have, and you obviously believe it. But, does the business know that? Even if they've heard it, do they believe it?

I doubt your business does. Frankly, I doubt most of your colleagues in the industry do. I still know far too many programmers who think that their only responsibility is to write code that creates features. If that were true, if we were nothing more than line workers delivering feature after feature, it would be plausible that productivity shouldn't suffer no matter where the factory exists.

But, it's not true. Being a software developer is about communication, collaboration, analysis and coding, at least. Being a phone call away instead of a face to face conversation away impacts communication and collaboration. Thus, productivity is impacted. There's no getting away from that.

How can you convince your business of that? I haven't solved that problem yet, unfortunately.

What could be
I do think Fred had the right ideas. He described scenarios that had previously benefited from distributed development, and what made those situations succeed.

The first suggestion is to get everyone together. You want the team to gel as one entity. Do version one entirely on-site if possible. However, don't do it with all local resources. Bring people from the desired off-site location to work on-site for the first release. The team members will build trust and friendships that last up to 6 months.

Once you are on version two you can move the desired team members back off-site. However, the travel isn't over for anyone. The off-site team should always have at least one member from the on-site team with them, likewise the on-site team should always have at least one member of the off-site team present. These aren't week long trips either. Each member of the team should visit the other location for a month, once every five to six months. That level of in-person communication should lead to high levels of trust and understanding.

The travel situation is even more drastic for the analysts and stakeholders. They should split their time between the two teams, if possible. Neither team should feel like the A team or the more important team. Any implication that one team is above the other team will lead to negative productivity impacts.

That might sound drastic, but it's the price of doing off-site development. The cost doesn't stop there.

Both team locations are going to need to invest heavily in infrastructure. The best video conferencing software and highly reliable bandwidth will also need to be purchased. The idea is to foster communication in every way possible. Without communication and collaboration, the project is doomed.

Open Source
Before anyone points out that it's hard to argue with the success of Open Source, I'd like to be clear -- I'm not. Open Source is obviously successful and developed most often in a distributed manner. However, there are a few differences that, I believe, make it a different situation entirely.

First of all, most people aren't paid to work on Open Source. When someone isn't paying you, you can often do whatever you feel like, whenever you feel like it. If someone is working in the same area of the code as you need to, you can just put off your changes until they are done.

However, the reality is that most people aren't usually working in common areas when they work on open source. Most open source projects are maintained by a few people who work within specific portions of the codebase. If changes need to happen in "your" portion of the codebase, you often queue them up to work on after you finish your current task.

Since there's little conflict between what you work on and what other team members work on there's significantly less communication and collaboration required, and what is necessary can happen at a much slower pace. If you need to make a change, it doesn't often need to happen right away. It can be put off until the team member on the other side of the world wakes up.

The codebase also evolves at a much slower pace. Six to ten people working in the same codebase 8 hours a day move much faster than the average Open Source project that sees 3 developers working a few hours a day.

Distributed development does work for Open Source, but that's not what I'm talking about.

Conclusion
I have heard of companies successfully doing remote pair programming and distributed development. One recipe I've heard is that everyone is off-site in different locations. I can see how that would work since it requires everyone to adopt a new work routine and make the best of it. While I believe it's possible to be successful, I think it's still bleeding edge at this point. You probably don't want to "try this at home" quite yet.

I think Distributed Development is probably the way of the future. As bandwidth and experience is more available the industry will continue to evolve in that direction. However, I think it's still probably about 5-10 years from being mainstream.

Passionate, Not Dogmatic

Ted Neward recently wrote a blog entry that began with the following text:

... the debates have begun, with all the carefully-weighed logic, respectful discourse, and reasoned analysis that we've come to expect and enjoy from this industry.

Yeah, right.

Ted's comment is funny because it's true. Ted's comment is also disappointing... because it's true.

In the past 3.5 years I had the opportunity to interact with some of the smartest people in our industry. I consider many of those smart people to be among the best software developers in the world. Unfortunately, some of the smart people I met weren't much more than assholes. The big difference I noticed between the two groups was -- The assholes were dogmatic, while the best developers were passionate

passionate: expressing, showing, or marked by intense or strong feeling
dogmatic: asserting opinions in a doctrinaire or arrogant manner
--dictionary.com

The difference between passionate and dogmatic is slim, but the result is dramatic.

Martin Fowler is a great example of someone who is passionate without being dogmatic. For example, Martin is a classicist, he prefers state based testing. However, in Mocks Aren't Stubs Martin examines both points of view and makes no assertion on which was is absolutely correct. That's a tough thing to do, but the result is a classic article that both mockists and classicists often refer to. That's just one example, but almost every article by Martin provides at least 2 points of view. The result is extremely valuable.

I used to be dogmatic. I have no problem admitting it. My earlier writing is clearly arrogant and often shortsighted. Part of the problem was lack of experience. When you take an immature industry and give a platform to someone with limited heuristics you are bound to receive solutions with limited applicability.

As I gained more experience I realized that what I considered to be best practices were only best practices within certain contexts. I also realized that presenting something as the "one true way" only benefited those that worked within exactly the same context that I worked. People who follow my advice when it doesn't apply to their context must fail. The advice isn't flawed, but it is incomplete. You need to see the full picture.

Two interesting things happen when you write passionate entries instead of dogmatic entries.

• Your advice is more widely and appropriately used.
• Your traffic goes down significantly.

On a recent podcast Joel Spolsky noted that most advice needs contextual information. Unfortunately, contextual information implies that the advice isn't universally applicable. While that's great for a small subset of professionals who are interested in best practices and improvement, the vast majority of people in our industry are still in search of silver bullets. The easiest way to ensure that your advice is missed by the majority of the industry is to spend the first 2 paragraphs of the entry describing the context.

Conversely, our industry loves dogmatic advice. For example, DRY is blindly, dogmatically followed. I'm not a fan of blindly following DRY, so I wrote about the value of duplication within tests. I attempted to give a counter point of view with contextual examples, but at the end of the day the entry got a large amount of traffic solely from the authoritative title.

It's easy to spot the difference between a dogmatic entry and a passionate entry. The dogmatic entry focuses on the best practice alone. However, a passionate entry gives equal weight to context and the best practice. Passionate entries are much more likely to see successful application, even if they don't make the top of reddit.

Preferring passionate to dogmatic entries is ultimately good, but you suffer in the short term. A career in software development is obviously a long term play, but you can't always blame someone for looking for short term gains. Of course, Martin Fowler is an example of the success you can achieve by sticking with passion over dogmatism.

There is one large upside to being passionate instead of dogmatic: You gain significantly more opportunities to learn. I can't count the number of times in the last year that I've said "I prefer the way I'm suggesting because I know it works, but let's do it your way and see if it's superior". (credit: I'm fairly sure I stole that phrase from George Malamidis while we worked together at TrafficBroker) Sometimes I was right, sometimes I was wrong, but I always learned something by trying a new approach.

That upside is what I believe truly separates the best in our industry from the assholes. The passionate leaders are constantly learning the best ways to do things, while the dogmatic leaders have stopped evolving their approach. As I said before, both groups are smart, but the dogmatic developers can only get by on their wisdom for so long. Eventually all dogmatic leaders become irrelevant.