Calling Clojure from Java

Calling Clojure from Java is easy, if you know which classes are important.

On my current project I make all my Clojure files resources, load them, and call the functions directly. The following example shows Clojure printing the argument it's given.

; printer.clj
(ns printer)

(defn print-string [arg]
  (println arg))

// Java calling code
RT.loadResourceScript("printer.clj");
RT.var("printer", "print-string").invoke("hello world");

There's a few things worth noting about the example: RT.var takes the namespace name and the function name. The Var returned by RT.var has an invoke method that allows you to pass any number of Objects. The invoke method also returns an Object, which allows you return values from Clojure where necessary.

It's also worth noting that the Clojure/Java interop is very, very good. You could pass a Java object to Clojure, make changes to it in Clojure, and return it back to Java. Of course, you might not even need to return it to Java since the instance referenced in Clojure would be the same instance referenced in Java.

Kill Your Darlings

When I worked with Zak Tamsen, one of his favorite (software development) sayings was: Kill Your Darlings. The idea was simple, don't get too attached to code.

I'm not sure why so many developers become so attached to code they've written. Perhaps developers have the same attachment to code that artists have to their paintings or music. Or, perhaps they view their legacy code as job security. Whatever the reason, developers seem to have some illogical desire to extend the life of code they've produced.

Of course, the desire to make legacy code live-on isn't universal. I tend to live in the opposite world, where I can't delete code fast enough. As much as I'd like to believe that people who share my views are evolved in some way, I know it isn't true. At least in my case, I like to delete code because I have ADD. Plain and simple.

Someone recently asked me how I deal with developers who are over-attached to their code. I've only found one successful way: create a finished solution that is simpler, and then share it with the original developer. Unfortunately, this path doesn't exactly promote collaboration, so it can definitely lead to other problems.

George Malamidis taught me something about code attachment a few years ago: You always gain by allowing someone to show you an alternative solution. If someone wants to solve a problem in a different way, there are several gains to be had. If their way is inferior, you have an opportunity to mentor a team-mate. If their way is equally elegant, you've gained another solution, or point of view that may be superior in the future. If their way is superior you learn something new and the codebase improves. In exchange for these gains you only need to give up time. Time is valuable, but it's also well spent on improving the ability of a team-mate or your personal ability.

Michael Feathers has also written on this topic, specifically focusing on frameworks. In Stunting a Framework, Michael discusses creating small focused frameworks and then letting them go. I think Michael really nailed it with that entry, it's definitely worth a quick read.

I think killing your darlings extends beyond codebases and frameworks to languages themselves. At SpeakerConf 2009, I floated the idea that we should more actively seek to kill languages. Perhaps, after 3 versions of a language, it's time for that language to be retired. Imagine what we could create if the resources dedicated to Java were instead focused on creating a successor to Java. Think of all the time that would be saved if backwards compatibility became a non-issue.

I can imagine the horror of developers who have written and used frameworks for Java. What would we do without all those frameworks? The reality is, we'd port those frameworks and improve them. There's thousands of developers who are dying to port an existing framework to a new language. And, frameworks should be stunted anyway, if you agree with Michael (which I do).

We are already evolving. New languages are appearing all around us. Frameworks are born and killed at a rapid pace. However, the attachment to code, frameworks, and languages only slows our maturing process. Be aware of, and support progress. Kill your darlings.

Retrospective Trust Level

Retrospectives can be complicated meetings. Done correctly, they can provide immense value. Done poorly, they can be a show that provides negative value. A well run retrospective requires more than just going through the motions. Several things contribute to a successful retrospective; however, in my experience the key ingredient is trust.

I expect Fred George (all the way on the right, 2nd from the top) would probably agree with me. Fred is the only person I've ever met that would start each retrospective by asking everyone to write down, on a scale of 1 to 5, where their trust level is at.

A retrospective without trust is pretty worthless, so measuring trust at the beginning of the meeting definitely makes sense. If the trust level is ever below the acceptable level (assuming 5 is complete trust, anything below a 4 is potentially a problem) then the meeting doesn't proceed until a solution is found to the lack of trust.

Of course, if trust is low, you might not give a true trust level. This problem can fairly easily be addressed by having a retrospective facilitator that is in no way invested in the project. The facilitator can collect the trust measurements in an anonymous manner that protects the innocent team members.

If you're finding that your retrospectives aren't providing as much value as they should be, you might want to measure the trust level. You may find that people are afraid to talk about the bigger problems.

Continual Retrospective

I'm a huge fan of retrospectives. When consulting I found retrospectives to be absolutely required. An enormous amount of value was derived from expressing concerns and discussing the expected and actual impacts. However, I've also worked on a few highly functioning teams that never seemed to get the same value from retrospectives.

Highly functioning teams tend to address issues as soon as they are discovered, which is far better, but removes some of the value that is usually derived from retrospectives.

I don't like ad-hoc retrospectives, I prefer a routinely scheduled meeting. I favor the scheduled version because often it doesn't seem like a retrospective is needed and it's delayed too long. Delaying valuable meetings definitely doesn't seem like a good plan.

The problem is, on highly functioning teams we'd often end up in the retrospective with only 1 or 2 issues to talk about. Eventually the retrospectives would seem superfluous and be removed from the calendar. I've always been uneasy with disregarding something that's proven so valuable in the past, but you can't argue with removing waste. (where waste = loss of time)

On highly functioning teams, I've always worried that smaller issues were falling through the cracks and costing us efficiency. But, retrospectives seemed too expensive a tool to identify the smaller issues.

A few years ago I worked on a team that had an "issues" white board. Anything that annoyed you could be put on the issues board. Any time you ran into an issue, you would add a new issue, or put a + next to the existing issue if the issue had already been recorded.

The issues board gave us great visibility into things that bothered the team. Adding plusses gave us a good gauge to see which issues were the most annoying. If something was annoying and encountered often, it would quickly gather severals plusses. However, if something was annoying, but extremely rarely encountered, it might not make sense to invest the effort necessary to address the issue.

As an example, if a test has a race condition that causes the build to fail 5 times a week, it's probably going attract plusses quickly and signal the team that the race condition needs to be addressed. Conversely, if the race condition causes the build to fail once a year, and it would take substantial effort to fix, it's not likely to get '+ support'. And, any issue without + support is probably not worth addressing. Usually, issues with little + support are removed after it appears that the ROI isn't positive on addressing the issue. If the issue pops back up it can easily be added back to the issues board.

The first time I used the issues board it was in conjunction with retrospectives; however, these days I find the issues board to be a great replacement for retrospectives on highly functioning teams. I view the issues board as a continual retrospective that can be modified at any time. You also get the added benefit of providing a real time view into what currently slows or stops progress on the team.

Replacing retrospectives with a continual retrospective isn't something I'd recommend under normal circumstances. Like I originally said, I find retrospectives to usually be extremely valuable. However, if the value of retrospectives seems to be negative when compared with the value of time, then a continual retrospective is probably better than no retrospective at all.

Thoughts on Developer Testing

This morning I read Joel: From Podcast 38 and it reminded me how immature developers are when it comes to testing. In the entry Joel says:

a lot of people write to me, after reading The Joel Test, to say, "You should have a 13th thing on here: Unit Testing, 100% unit tests of all your code."

At that point my interest is already piqued. Unit Testing 100% of your code is a terrible goal and I'm wondering where Joel is going to go with the entry. Overall I like the the entry (which is really a transcribed discussion), but two things in the entry left me feeling uneasy.

• Joel doesn't come out and say it, but I got the impression he's ready to throw the baby out with the bath water. Unit testing 100% of your code is a terrible goal, but that doesn't mean unit testing is a bad idea. Unit testing is very helpful, when done in a way that provides a positive return on investment (ROI).
• Jeff hits it dead on when he says:

  ...what matters is what you deliver to the customer...

  Unfortunately, I think he's missing one reality: Often, teams don't know what will make them more effective at delivering.

I think the underlying problem is: People don't know why they are doing the things they do.

A Painful Path
Say you read Unit Testing Tips: Write Maintainable Unit Tests That Will Save You Time And Tears and decide that Roy has shown you the light. You're going to write all your tests with Roy's suggestions in mind. You get the entire team to read Roy's article and everyone adopts the patterns.

All's well until you start accidently breaking tests that someone else wrote and you can't figure out why. It turns out that some object created in the setup method is causing unexpected failures after your 'minor' change created an unexpected side-effect. So, now you've been burned by setup and you remember the blog entry by Jim Newkirk where he discussed Why you should not use SetUp and TearDown in NUnit. Shit.

You do more research on setup and stumble upon Inline Setup. You can entirely relate and go on a mission to switch all the tests to xUnit.net, since xUnit.net removes the concept of setup entirely.

Everything looks good initially, but then a few constructors start needing more dependencies. Every test creates it's own instance of an object; you moved the object creation out of the setup and into each individual test. So now every test that creates that object needs to be updated. It becomes painful every time you add an argument to a constructor. Shit. Again.

The Source of Your Pain
The problem is, you never asked yourself why. Why are you writing tests in the first place? Each testing practice you've chosen, what value is it providing you?

Your intentions were good. You want to write better software, so you followed some reasonable advice. But, now your life sucks. Your tests aren't providing a positive ROI, and if you keep going down this path you'll inevitably conclude that testing is stupid and it should be abandoned.

Industry Experts
Unfortunately, you can't write better software by blindly following dogma of 'industry experts'.

First of all, I'm not even sure we have any industry experts on developer testing. Rarely do I find consistently valuable advice about testing. Relevance, who employs some of the best developers in the world, used to put 100% code coverage in their contracts. Today, that's gone, and you can find Stu discussing How To Fail With 100% Code Coverage. ObjectMother, which was once praised as brilliant, has now been widely replaced by Test Data Builders. I've definitely written my fair share of stupid ideas. And, the examples go on and on.

We're still figuring this stuff out. All of us.

Enlightenment
There may not be experts on developer testing, but there are good ideas around specific contexts. Recognizing that there are smart people with contextually valuable ideas about testing is very liberating. Suddenly you don't need to look for the testing silver-bullet, instead you have various patterns available (some conflicting) that may or may not provide you value based on your working context.

Life would be a lot easier if someone could direct you to the patterns that will work best for you, unfortunately we're not at that level of maturity. It's true that if you pick patterns that don't work well for your context, you definitely wont see positive ROI from testing in the short term. But, you will have gained experience that you can use in the future to be more effective.

It's helpful to remember that there aren't testing silver-bullets, that way you wont get lead down the wrong path when you see someone recommending 100% code coverage or other drastic and often dogmatic approaches to developer testing.

Today's Landscape
Today's testing patterns are like beta software. The patterns have been tested internally, but are rarely proven in the wild. As such, the patterns will sometimes work given the right context, and other times they will shit the bed.

I focus pretty heavily on testing and I've definitely seen my fair-share of test pain. I once joined a team that spent 75% of their time writing tests and 25% of their time delivering features. Not a member of the team was happy with the situation, but the business demanded massively unmaintainable Fit tests.

Of course, we didn't start out spending 75% of our time writing Fit tests. As the project grew in size, so did the effort needed to maintain the Fit tests. That kind of problem creeps up on a team. You start by spending 30% of your time writing tests, but before you know it, the tests are an unmaintainable mess. This is where I think Jeff's comments, with regard to writing tests that enable delivery, fall a bit short. Early on, Fit provided positive ROI. However, eventually, Fit's ROI turned negative. Unfortunately, by then the business demanded a Fit test for every feature delivered. We dug ourselves a hole we couldn't get out of.

The problem wasn't the tool. It was how the process relied on the Fit tests. The developers were required to write and maintain their functional tests using Fit, simply because Fit provided a pretty, business readable output. We should have simply created a nice looking output for our NUnit tests instead. Using Fit hurt, because we were doing it wrong.

The current lack of maturity around developer testing makes it hard to make the right choice when picking testing tools and practices. However, the only way to improve is to keep innovating and maturing the current solutions.

If It Hurts, You're Doing It Wrong
Doing it right is hard. The first step is understanding why you use the patterns you've chosen. I've written before about the importance of context. I can explain, in detail, my reasons for every pattern I use while testing. I've found that having motivating factors for each testing pattern choice is critical for ensuring that testing doesn't hurt.

Being pragmatic about testing patterns also helps. Sometimes your favorite testing pattern wont fit your current project. You'll have to let it go and move on. For example, on my current Java project each test method has a descriptive name. I maintain that (like methods and classes) some tests are descriptive enough that a name is superfluous, but since JUnit doesn't allow me to create anonymous test methods I take the path of least resistance. I could write my own Java testing framework and convince the team to use it, but it would probably hurt. The most productive way to test Java applications is with JUnit, and if I did anything else, I'd be doing it wrong.

I can think of countless examples of people doing it wrong and dismissing the value of a contextually effective testing pattern. The biggest example is fragile mocking. If your mocks are constantly, unexpectedly failing, you're doing something wrong. It's likely that your tests suffer from High Implementation Specification. Your tests might be improved by replacing some mocks with stubs. Or, it's possible that your domain model could be written in a superior way that allowed more state based testing. There's no single right answer, because your context determines the best choice.

Another common pain point in testing is duplicate code. People go to great lengths to remove duplication, often at the expense of readability. Setup methods, contexts, and helper methods are all band-aids for larger problems. The result of these band-aids is tests that are painful to maintain. However, there are other options. In the sensationally named entry Duplicate Code in Your Tests I list 3 techniques that I've found to be vastly superior to setup, contexts and helper methods. If those techniques work for you, that's great. If they don't, don't just shove your trash in setup and call it a day. Look for your own testing innovations that the rest of us may benefit from.

If something hurts, don't look for a solution that hurts slightly less, find something that is a joy to work with. And, share it with the rest of us.

Tests Should Make You More Effective
What characterizes something as 'effective' can vary widely based on your context.

Some software must be correct or people die. This software obviously requires thorough testing. Other software systems are large and need to evolve at a fairly rapid pace. Delivering at a rapid pace while adding features almost always requires a fairly comprehensive test suite, to ensure that regression bugs don't slip in.

Conversely, some software is internal and not mission critical. In that case, unhandled exceptions aren't really a big deal and testing is clearly not as high a priority. Other systems are small and rewritten on a fairly consistent basis, thus spending time on thorough testing is likely a waste. If a system is small, short lived, or less-important, a few high level tests are probably all you'll really need.

All of the example environments and each other type of environment share one common trait: You should always look at your context and see what kind of tests and what level of testing will make you more effective.

Tests Are Tools
The tests are really nothing more than a means to an end. You don't need tests for the sake of having tests, you need malleable software, bullet-proof software, internal software, or some other type of software. Testing is simply another tool that you can use to decrease the amount of time it takes to get your job done.

Testing can help you-

• Design
• Protect against regression
• Achieve sign-off
• Increase customer interaction
• Document the system
• Refactor confidently
• Ensure the system works correctly
• ...

Conclusion
When asking how and what you should test, start by thinking about what the goal of your project is. Once you understand your goal, select the tests that will help you achieve your goal. Different goals will definitely warrant using different testing patterns. If you start using a specific testing pattern and it hurts, you're probably using a pattern you don't need, or you've implemented the pattern incorrectly. Remember, we're all still figuring this out, so there's not really patterns that are right; just patterns that are right in a given context.

[Thanks to Jack Bolles, Nat Pryce, Mike Mason, Dan Bodart, Carlos Villela, Martin Fowler, and Darren Hobbs for feedback on this entry]