Tuesday, February 28, 2012

Aligning Business & Programmer Goals

A theme emerged, last week at speakerconf, around the idea that programmers and businesses still don't seem to be on the same page. I'm not sure if it came up even earlier, but for me it began after I gave a presentation on Lessons Learned Adopting Clojure. The talk is similar to Lessons Learned while Introducing a New Programming Language, but focuses more on the technical aspects of selecting Clojure. Upon talk completion the first question I got was from Brian Goetz, something along the lines of:
(Brian) So, you wanted to use another language and simply introduced it into production?
(me) More or less.
(Brian) Do you think that's the most effective way for organizations to select languages or frameworks?
The first response to the question was the obvious and appropriate one:
That sounds better than a CTO making a decision while drunk and on a golf-course with a vendor. -- Josh Graham
Of course, I agree with that response, but that doesn't mean that my choice was the best answer either. The question is actually something I've been struggling with for awhile. At speakerconf Aruba 2011, Lyle (DRW CTO) gave a presentation on how we do things at DRW. One of Lyle's slides listed each language that we use in production at DRW. There were a lot of languages, probably around 25, and Dave Thomas questioned whether it was a positive or a maintenance nightmare. The answer today is the same as it was then, so far the ROI is positive - but, I couldn't help feeling uneasy about the situation.

I answered somewhat noncommittally, along the lines of: While selecting a language, I chose one that I felt anyone in DRW could easily learn and use. Additionally, I selected a language that aligned with our business goals of developing software quickly and delivering applications that performed at the speeds required by the traders. And, if not the man on the ground, who is better suited for selecting languages to introduce into production?

Brian followed up with:
The point I was trying to get at was: how does this scale? If every developer chooses what is best for them, is that necessarily best for the organization? How do we balance between the business value of increasing individual productivity and the business value of reducing maintenance costs or development risk? If we have to make compromises to achieve better scale, who should make these decisions, and how?

The usual developer argument -- "this is more productive for me, and increasing my productivity is good for the company" is good as far as it goes, but most developers incorrectly assume these interests are 100% aligned when they are in reality probably only merely overlapping.
These are truly great questions, and no one had a great response. The discussion basically died off as we pondered the tough answers to these questions.

At lunch the next day I asked Brian how he thought companies should go about selecting languages, and I think his response was spot-on (again, going from memory, so likely misquoting): I'm not sure, but I sure do see a lot of developers introducing whatever language or framework they want, leaving within a year, and sticking their employer with a big maintenance problem. The same can be said for consultants. They often show up advocating a sexy new language, mostly deliver a new application and then leave the company with an application that can't be maintained by the existing full-time staff.

Someone quickly spoke up, stating that Resume Driven Development has been going on for years. I can't argue with that, but I did have a different perspective.

A few years ago I joined DRW, and a substantial portion of my salary became bonus money. I had previously worked in places where getting bonus money was about as likely as meeting your future wife at a JavaOne, so I was obviously skeptical. Luckily, I had a friend who already worked for DRW; he reassured me that, in finance, having a bonus as part of your compensation was common, and so was it actually getting paid. I took the job, and instantly my performance at work was more directly tied to my income. I'm sure this reality influenced my decision making, and I witnessed the difference in the decisions my colleagues were making. When people spoke about using/upgrading frameworks/libraries, we didn't talk about what was cool, we discussed the risks of breaking things, the outstanding bugs with the framework/lib, and the immediate value that the inclusion would provide.

Given this experience, I advocated to the other speakerconf presenters that bonuses are a potential solution to combatting Resume Driven Development. The other speakerconf presenters gave me the same response I generally get when advocating bonuses for programmers: That works for you and your trading firm, but most people can't directly tie what they do to how the business makes money. I'm always skeptical of this response for 2 reasons: The business must be able to track whether or not their new idea is making them money, or, if the new software isn't targeted to an external customer, then it should be able to be rated by internal customers. Also, shouldn't the CTO or the Director of Software be able to make an informed bonus decision by either directly interacting with the teams, or trusting the feedback of the managers of those teams?

I guess, if you can't tie what you are doing to the value it's providing the company, and you don't trust your managers to give an accurate assessment, building your resume is probably the best use of your time... and the results aren't very surprising... but, I digress.

I gave the same response I've given a few dozen times: I just don't buy it. You should be able to see how many new customers your new website generated, or less cancellations, new reservations, policy efficiency, auction traffic, add-on sales purchased, etc. Each of the projects I've worked on for the past 8 years could generate metrics that I would have been happy to tie to a bonus. In our back-office, it's harder to tie the value of a back office system to an individual trade; however, we can capture metrics on uptime, latency, and any other attribute that we deem important to the internal systems that are customers of the back office systems.

If you want to measure your value and you trust your management, receiving part of your salary as a bonus is completely viable.

Aligning the goals of the business and the programmers came up again when the afternoon presentations began. Joshua Kerievsky is looking to tackle this same issue at Industrial Logic. Joshua's presentation discussed bridging the gap between managers and programmers, which is a more general version of the same conversation we'd been having about language adoption. Bridging this gap is something he's begun focusing on at Industrial Logic. The benefits are obvious for customers, they should receive higher quality and more relevant software. However, it's not just about customers, the programmers will be more effective and should also have higher job satisfaction rates. No one likes being stuck on yet another, doomed to fail, misguided project using outdated technology. Both parties should benefit by unifying the goals of the programmer and the business.

Josh knew my opinion on using bonuses to help align both parties; however, he and I both incorrectly assumed that the developers at Forward also received business performance based bonuses. I believe that they have some things in the works and situations can vary, but Fred George made it clear that in general a bonus is not tied to business focus. That's fairly standard, but what makes Forward unique is that the business works very, very closely with the programmers. I think Fred nicely summed up the situation: Our programmers are gamers, and we show them the P&L. They want to see that number grow, even if it doesn't directly deposit in their account.

Fred's statement rang very true to me. I've had jobs in the past where I didn't even have a bonus, but the ones that I felt most dedicated to were the ones where I was given visibility into the business. If I can't see the outcome of my programming choices to the business, I'll have a much harder time iterating towards success - worse, I won't even know at what level we are failing and I'll lack the motivation that the failure would provide me.

At the end of the day, programmers like to measure and improve. If you give them a P&L and let them know that it's the most important thing, they'll do whatever they can to help. If you don't give them a P&L they'll focus on making other metrics more efficient. This may include using new technologies to inspire more blog entires, spending hours unnecessarily tweaking the development process, or any other way of measuring their "success." The answer is obvious, let the programmers measure success the same way the business does, using the bottom line. What's not always obvious is how to properly share that information; however, all the discussion at speakerconf definitely convinced me that it's worth solving this issue even if it requires a bit of extra effort.

6 comments:

  1. I recommend Dan Pink's work on motivation: it turns out that having a strong financial incentive destroys creativity - our monkey brains focus in on concrete solutions most likely to win us the banana, and creativity suffers. That works very well when pressing buttons on an assembly line but not very well when metaprogramming, contemplating design patterns, or puzzling out better algorithms.
    Programmers should be paid enough that they don't think about the money any more, then rewarded by excellent working conditions. Bonuses and profit share are poison.

    ReplyDelete
  2. Sorry: http://www.ted.com/talks/dan_pink_on_motivation.html

    ReplyDelete
  3. Unfortunately, the calculation of bonuses at many companies leaves a lot to be desired, and doesn't really help align goals.

    Our dev team is treated as part of the business by our biz managers, they ask for our input into business priorities. At one point we decided to budget time over several weeks for each of us to go sit with some of the business people, learn how they do their jobs, and come back and explain it to the team, this way we covered all areas of the business, whether they were areas that used our application or not. We were able to identify many ways to help the business people with some simple automation solutions. We also understood the business domain more and became much better at delivering the features they actually wanted.

    This makes us more efficient, but it also helps us enjoy our work more. We spend less time fixing mistakes made by users, and more time developing new stuff to delight our customers.

    ReplyDelete
  4. Jay, I really enjoyed the post, but I feel like the most obvious and direct solution is being avoided, and has worked wonderfully for me.

    The bridge between management and engineering is simply metrics. At Tutorspree (where I'm VP of Engineering), we focus on building a winning culture to attract talent and deliver great solutions. Part of that culture is validation and choosing the best tool for the job.

    If your developers are committed to showing objective data why introducing a new technology is a better option (read better ROI), than using something in the stack, then everyone is happy. Developers get freedom, room to grow, responsibility is distributed, everyone's conversation is around a concrete set of metrics and data points, and there's a set expectation of performance (for some definition of performance as it relates to the metrics).

    Bonus pay for you is just an incentive to be objective in your analysis, but I agree with the previous comments, bonus pay isn't always the best option to build a culture around.

    ReplyDelete
  5. I work as a contractor part time and I have been very tempted to use Clojure in an ETL project for one of my clients, however I have resisted as I know they would have trouble finding a Clojure developer in this city.

    I think the approach is to use the "sexy new language" on some experimental, non-business-critical, application and use that as a way of selling the language to you clients. If the application exceeds expectations they wouldn't mind if you started using it in more important areas.

    ReplyDelete
  6. I am currently considering using Clojure - to which I'm still quite new - for an ETL 'problem'.

    The ETL tools we have (and have had) are awful to work with, and writing this in Java, our bread-and-butter language would mean endless wrapping and unwrapping and plumbing code that is just not all that useful for this particular problem. Of course, I'm going back-and-forth whether I should force my colleagues to use a programming language instead of an ETL tool (easy sell there given our current problems maintaining the tool) and whether if that would be a language, I should pick Clojure. Seems like the best tool for the job, but then again it can be hard to pick up if you haven't done LISP before.

    Anyway, I had to laugh at your line: I had previously worked in places where getting bonus money was about as likely as meeting your future wife at a JavaOne. Well, I've only once gotten a bonus worth mentioning, but did in fact meet my wife at JavaOne, believe it or not!

    ReplyDelete

Note: Only a member of this blog may post a comment.