Sharing Blog Example Code

I've been blogging since 2005, using various tools for publishing readable code. Originally you'd just put code out there, no colors, just a pre tag and some indention. Then 'export to html' became pretty standard in various editors, and everyone's life got better. These days, people want code examples available to play with - and I use GitHub to make that possible.

I've been toying with different formats for sharing code, and my last blog entry demonstrates what I've settled on.

• Each blog entry with a non-trivial amount of code will live in a repository owned by blog-jayfields-com
• Commits will contain working code that can be executed (if possible)
• I'll use gist-it to embed code that can be found in the GitHub repo
• I'll create a gh-pages index.html with the content of the blog entry, for people to review and provide feedback.

Hopefully the extra integration with GitHub will make it easier to try out any code that I publish. As always, please drop me a line in the comments if you have any suggestions.

Reading Clojure Stacktraces

Clojure stacktraces are not incredibly user friendly. Once I got used to the status quo, I forgot how much noise lives within a stacktrace; however, every so often a Clojure beginner will remind the community that stacktraces are a bit convoluted. You can blame the JWM, lack of prioritization from the Clojure community, or someone else if you wish, but the reality is - I don't expect stacktraces to change anytime soon. This blog entry is about separating the signal from the noise within a stacktrace.

note: all code for this blog entry can be found at: http://github.com/blog-jayfields-com/Reading-Clojure-Stacktraces

Let's start with a very simple example.


Running (I'm using 'lein run') the above code you should get a stacktrace that looks like the output below.

lmp-jfields03:reading-clojure-stacktraces jfields$ lein run
Exception in thread "main" java.lang.RuntimeException: thrown
 at reading_clojure_stacktraces.core$foo.invoke(core.clj:3)
 at reading_clojure_stacktraces.core$_main.invoke(core.clj:6)
 at clojure.lang.Var.invoke(Var.java:397)
 at user$eval37.invoke(NO_SOURCE_FILE:1)
 at clojure.lang.Compiler.eval(Compiler.java:6465)
 at clojure.lang.Compiler.eval(Compiler.java:6455)
[blah blah blah]

I snipped a fair bit of stacktrace and replaced it with [blah blah blah]. I did that because that's what I mentally do as well, I look for the last line that includes a file that I've created and I ignore everything after a few lines below my line. That is my first recommendation - If you see a stacktrace, it's likely that the problem is in your code, not Clojure. Look for the last line of your code (N) and ignore every line below N + 3.

In this example, user$eval... likely has something to do with lein, and I can safely assume that the problem is likely not in there. Moving up from there I can see a line from my code:

reading_clojure_stacktraces.core$_main.invoke(core.clj:6)

When I read the above line I see the problem is in namespace 'reading-clojure-stacktraces/core', in the function '-main', in the file core.clj, on line 6. I'm no Clojure internals expert, but I believe Clojure actually creates a class named reading_clojure_stacktraces.core$_main with an 'invoke' method; however, I truthfully don't know (and you wont need to either). Whether a class is created or not, it makes sense that the line will need to be formatted to fit a valid Java class name - which explains why our dashes have been converted to underscores.

Moving up another line, I can see that the issue is likely inside the 'foo' function of the reading-clojure-stacktraces namespace. A quick review of the original code shows that line 3 of core.clj contains the call to throw, and everything makes perfect sense.

If all Clojure stacktraces were this simple, I probably wouldn't bother with this blog entry; however, things can become a bit more complicated as you introduce anonymous functions.

The following snippet of code removes the 'foo' function and throws an exception from within an anonymous function.

Another trip to 'lein run' produces the following output.

Exception in thread "main" java.lang.RuntimeException: thrown
 at reading_clojure_stacktraces.core$_main$fn__9.invoke(core.clj:4)
 at reading_clojure_stacktraces.core$_main.invoke(core.clj:4)
 at clojure.lang.Var.invoke(Var.java:397)
 at user$eval38.invoke(NO_SOURCE_FILE:1)
 at clojure.lang.Compiler.eval(Compiler.java:6465)

The above stacktrace does give you the correct file and line number of where the issue originates; however, you'll notice that the function that threw the exception has become a bit less easy to identify. My use of an anonymous function led to Clojure naming the function fn__9, and there's nothing wrong with that. In fact, this example is especially readable as the stacktrace shows that fn__9 was created inside the -main function.

I expect you'd be able to find the issue with our contrived example without any further help; however, production code (often making use of high order functions) can lead to significantly more complex stacktraces. You could forsake anonymous functions, but there's a nice middle ground that is also helpful for debugging - temporarily name your anonymous functions.

Clojure's reader transforms the Anonymous function literal in the following way.

#(...) => (fn [args] (...))

Therefore, the following code will be the same as the example above, from Clojure's perspective.

Another quick 'lein run' verifies that the stacktrace is the same (and I see no reason to repeat it here). However, now that we've switched to fn, we can provide a (rarely used, optional) name.

At this point, 'lein run' should produce the following output.

Exception in thread "main" java.lang.RuntimeException: thrown
 at reading_clojure_stacktraces.core$_main$i_throw__9.invoke(core.clj:4)
 at reading_clojure_stacktraces.core$_main.invoke(core.clj:4)
 at clojure.lang.Var.invoke(Var.java:397)
 at user$eval38.invoke(NO_SOURCE_FILE:1)
 at clojure.lang.Compiler.eval(Compiler.java:6465)

Now our line contains a bit more information. The two $ signs still indicate that the function with an issue is a function created inside -main; however, our stacktrace also includes the name (in bold) we specified for our function. You can use any valid symbol characters, so feel free to put anything you want in the name while you're debugging.

note: Symbols begin with a non-numeric character and can contain alphanumeric characters and *, +, !, -, _, and ? -- clojure.org.

So far, all of the examples have been somewhat noisy, but mildly easy to mentally filter. Unfortunately, idiomatic Clojure code can also lead to stacktraces that bounce back and forth between your code and the standard library, leaving you to sift through significantly longer stacktraces.

The following snippet of code throws a NullPointerException due to a mistake I clearly made, but the last line of 'my code' is in the lower half of a long stacktrace.

The above example code produces the below stacktrace.

Exception in thread "main" java.lang.NullPointerException
 at clojure.lang.Numbers.ops(Numbers.java:942)
 at clojure.lang.Numbers.inc(Numbers.java:110)
 at clojure.core$inc.invoke(core.clj:862)
 at clojure.core$map$fn__3811.invoke(core.clj:2430)
 at clojure.lang.LazySeq.sval(LazySeq.java:42)
 at clojure.lang.LazySeq.seq(LazySeq.java:60)
 at clojure.lang.RT.seq(RT.java:466)
 at clojure.core$seq.invoke(core.clj:133)
 at clojure.core$print_sequential.invoke(core_print.clj:46)
 at clojure.core$fn__4990.invoke(core_print.clj:140)
 at clojure.lang.MultiFn.invoke(MultiFn.java:167)
 at clojure.core$pr_on.invoke(core.clj:3264)
 at clojure.core$pr.invoke(core.clj:3276)
 at clojure.lang.AFn.applyToHelper(AFn.java:161)
 at clojure.lang.RestFn.applyTo(RestFn.java:132)
 at clojure.core$apply.invoke(core.clj:600)
 at clojure.core$prn.doInvoke(core.clj:3309)
 at clojure.lang.RestFn.applyTo(RestFn.java:137)
 at clojure.core$apply.invoke(core.clj:600)
 at clojure.core$println.doInvoke(core.clj:3329)
 at clojure.lang.RestFn.invoke(RestFn.java:408)
 at reading_clojure_stacktraces.core$_main.invoke(core.clj:7)
 at clojure.lang.Var.invoke(Var.java:397)
 at user$eval37.invoke(NO_SOURCE_FILE:1)
 at clojure.lang.Compiler.eval(Compiler.java:6465)
 at clojure.lang.Compiler.eval(Compiler.java:6455)

In situations like these, I generally look at the top few lines to get a bit of context, and then I scroll down to find the last line of 'my code'. Looking at the top 4 lines from the stacktrace I can see that the issue is with the inc function, which was passed to the high order function named map. If I look father down the stacktrace, I can see that line 7 in reading-clojure-stacktraces.core is where the issue began in 'my code'.

If you look at line 7 of (reading-clojure-stacktraces) core.clj, you'll notice that I'm merely printing the results of calling foo - yet the issue seems to be with the map function that is invoked within foo. This is because map is lazy, and the evaluation is deferred until we attempt to print the results of mapping inc. While it's not exactly obvious, the stacktrace does contain all the hints we need to find the issue. Line 3 lets us know that inc is getting a nil. Line 4 lets us know that it's happening inside a map. Line 5 lets us know that we're dealing with laziness. And, the line containing our namespace lets us know where to begin looking.

The following example is very similar; however, it uses a partial to achieve the same result.

The above example code produces the below stacktrace.

Exception in thread "main" java.lang.NullPointerException
 at clojure.lang.Numbers.ops(Numbers.java:942)
 at clojure.lang.Numbers.add(Numbers.java:126)
 at clojure.core$_PLUS_.invoke(core.clj:927)
 at clojure.lang.AFn.applyToHelper(AFn.java:163)
 at clojure.lang.RestFn.applyTo(RestFn.java:132)
 at clojure.core$apply.invoke(core.clj:602)
 at clojure.core$partial$fn__3794.doInvoke(core.clj:2341)
 at clojure.lang.RestFn.invoke(RestFn.java:408)
 at clojure.core$map$fn__3811.invoke(core.clj:2430)
 at clojure.lang.LazySeq.sval(LazySeq.java:42)
 at clojure.lang.LazySeq.seq(LazySeq.java:60)
 at clojure.lang.RT.seq(RT.java:466)
 at clojure.core$seq.invoke(core.clj:133)
 at clojure.core$print_sequential.invoke(core_print.clj:46)
 at clojure.core$fn__4990.invoke(core_print.clj:140)
 at clojure.lang.MultiFn.invoke(MultiFn.java:167)
 at clojure.core$pr_on.invoke(core.clj:3264)
 at clojure.core$pr.invoke(core.clj:3276)
 at clojure.lang.AFn.applyToHelper(AFn.java:161)
 at clojure.lang.RestFn.applyTo(RestFn.java:132)
 at clojure.core$apply.invoke(core.clj:600)
 at clojure.core$prn.doInvoke(core.clj:3309)
 at clojure.lang.RestFn.applyTo(RestFn.java:137)
 at clojure.core$apply.invoke(core.clj:600)
 at clojure.core$println.doInvoke(core.clj:3329)
 at clojure.lang.RestFn.invoke(RestFn.java:408)
 at reading_clojure_stacktraces.core$_main.invoke(core.clj:7)
 at clojure.lang.Var.invoke(Var.java:397)
 at user$eval37.invoke(NO_SOURCE_FILE:1)
 at clojure.lang.Compiler.eval(Compiler.java:6465)

Again, you'll want to skim the stacktrace for hints. In the above stacktrace we can see that line 3 is telling us that + is the issue. Line 7 lets us know that partial was used. And, the remaining hints are the same as the previous example.

Skimming for hints may look painful at first. However, you quickly learn to filter out the common Clojure related noise. For example, anything that starts with 'clojure' and looks like a standard Java class name is highly unlikely to be where a problem exists. For example, clojure.lang.Numbers.ops isn't likely to have a bug. Likewise, you'll often see the same classes and methods repeated across all possible errors - clojure.lang.AFn, clojure.lang.RestFn, clojure.core$apply, clojure.lang.LazySeq, clojure.lang.RT, clojure.lang.MultiFn, etc, etc. These functions are often used building blocks for almost everything Clojure does. Those lines provide a bit of signal, but (for the most part) can safely be ignored.

Again, it can be a bit annoying to deal with Clojure stacktraces when getting started; however, if you take the time to understand which lines are signal and which lines are noise, then they become helpful debugging tools.

related: If you want a testing library that helps you filter some of the stacktrace noise, you might want to check out expectations.

Clojure: Production Web REPL

A REPL is a powerful tool. I use the REPL extensively in development, and I recently added a web REPL to each of our production applications.

Context
A production REPL is not for the faint of heart. I wouldn't add a REPL to production in 95% of the projects I've been a part of; however, I'm currently working with two other developers that I completely trust with the power of a production REPL. As a simple rule, I wouldn't give access to a production REPL to anyone that doesn't also have root access to the server - if you can't be trusted with one, it's likely that you can't be trusted with the other. However, if you can be trusted, I'd rather you have all of your tools available to you.

sidebar: the most interesting story I've heard about a REPL in prod was the following - Debugging a program running on a $100M piece of hardware that is 100 million miles away is an interesting experience. Having a read-eval-print loop running on the spacecraft proved invaluable in finding and fixing the problem. -- Lisping at JPL

Motivation
Don't get the wrong impression, I'm not regularly redefining functions in production. I rarely use a prod REPL, but when I do, I almost always use it to read reference data. On the very rare occasion, I'll wrap an existing function with a new version that logs the value of the incoming arguments. In short, I use the prod REPL (again, sparingly) to gather more data, not to fix outstanding bugs. I'm not saying I would never use it to fix a bug, but that would be an extremely exceptional case.

Code
It's actually very easy to plug a web REPL into a Clojure application that already has a web interface. The Clojure applications I work on tend to have web sockets, and the following solution uses web sockets; however, there's no reason that you couldn't use simple web requests and responses. Lastly, I didn't actually write this code. Someone at DRW wrote it (I have no idea who) and it's been copy-pasted several times since then. Without further adieu, the code:

Signs That Your Project Team Might Be Too Large

Amanda Laucher recently queried the twitterverse on the topic of team size.

In my opinion, my last project team was "too large". This blog entry will focus on the things that happened that drove me to that conclusion.

note: I was the only person on the team who believed it was too large. The rest of the team thought we 'might' be too large, and the cost of making a change was greater than the pain of remaining 'potentially' too large. Since I was the minority, it's safe to assume that my opinion is controversial at a minimum.

In many ways my last project was a very, very well funded start-up. We started small, with 3 developers; however, as the scope of our problem grew so went the size of our team. When the team had reached 7 (5 developers in one office and 2 in another), I began to advocate for a team split. By the time we reached 10 total, I was positive that a split was appropriate. As I stated, we were in start-up mode. I wanted to deliver as fast as possible and I felt that several factors were impacting our ability to produce.

Camps. Technology choices originally drove my desire to split the team. The project had speed requirements that forced the use of Java for some portions of our solution; however, other portions of the codebase could have been written using Java, Clojure, or even Ruby. I held the opinion that anything that could be written in Clojure should be written in Clojure. I felt faster writing code in Clojure (vs Java) and I wanted to deliver as fast as I possibly could. Two of my colleagues agreed with my opinion, 2 abstained, and 2 strongly disagreed with me. The two that disagreed with me thought it was easier to write everything in one language, using one toolset. I completely see their point of view, and acknowledge all of their opinions. They are not wrong, we just have incompatible opinions on software.

I need to be clear here, these were good developers. It would have been easy if they were inferior developers - we would have improved their skills or found them a new home. However, that wasn't the case. These were guys that consistently delivered as much, if not more value than I did - they just wanted to get things done is a way that clashed with my approach. The team was more or less split into 2 camps - pro Clojure and pro Java.

Countless hours were spent discussing how to deal with this difference of opinions. Retrospectives were often composed of 2 types of problems: Problems that we all agreed on and solved, and problems that we would never agree on and never solved. The 2nd types of problems appeared at every retrospective and always led to the same interaction.

• someone (usually me) bitched about the lack consistency on the team
• people from my camp agreed that it was an issue, and spoke in generic terms (not wanting to target or alienate anyone).
• people from the opposing camp made their (completely valid) points.
• the room became awkwardly silent; everyone knew we were at an impass
• someone made a joke to ease tensions
• we moved on, no resolution

This situation infuriated me, but, realistically, there was never going to be a solution that made the entire team happy. Everything that was written in Clojure annoyed the pro Java camp, and anything that wasn't performance sensitive and was written in Java annoyed the pro Clojure camp. I naively believed that the pro Java guys would "see the light", and tried everything I could think of to convince them that Clojure was the right choice - and it was for me, but it (likely) never will be for them. In the end, I decided that advocating Clojure was a waste of time and focused my efforts on getting the team split by compatible opinions.

reviewer feedback

On Jun 18, 2012, Patrick Farley wrote:

I think your points about non-compatible opinions on a team, in the saddest case, can occur on a team of two. Do you suppose two distinct camps (as opposed to a bickering mob) become more likely as team size grows?

I do believe it is more likely that camps occur as the team sizes grow. I believe disagreements are healthy; however, camps are counterproductive - due to too much time spent in arbitration. I do agree that you can have camps on a team size two, and it's possible to have no camps on a team of size 10 - which is why I don't think size X will guarantee camps, but that if you have camps then X might be too many.

If your team has camps with incompatible (yet, perfectly valid) opinions then it's a sign that it's possibly too big, or it has members that would be more effectively used if swapped with someone else in the organization that holds compatible opinions. Or, more precisely, if you have camps (where camp is defined as N or more people) with conflicting opinions, and you have a solution that can be split vertically or horizontally, then your team size may be "too big" - where too big is defined as "a split would likely increase the productivity of the members of both camps". It could be possible to replace X split-away members with Y new members where (Y > X) and as long as you do not have camps, the team would not be considered too big - which is why you cannot simply state that Z people on a team is 'too big'. note: N is defined as the number of people required to reasonably address bus risk. On a highly functional team with almost no turnover, N can equal 2. More often, N will be 3 or more.

Depth Shortage. Agile Enthusiasts loved to point out the benefits of Collective Code Ownership - and they are undeniable when contrasted with Strong Code Ownership. However, I believe that Collective Code Ownership actually hindered the delivery pace of the previously referenced development team. Collective Code Ownership combined with our team size resulted in many team members having breadth of knowledge, but lacking depth of knowledge of the system.

reviewer feedback

On Jun 18, 2012, J. B. Rainsberger wrote:

Do you believe that CCO hindered the pace of delivery, or that the code base grew to a size and level of complication that made it hard to own collectively?

Definitely the latter. Our problem had grown to a size that caused CCO to go from a positive impact to a negative one. This was not due to a flaw in CCO, this was due to applying CCO to a situation where it was no longer the most effective choice.

The problem we were solving had 3rd party, distributed system, compliance, extreme performance, and algorithmic concerns that all needed attention. On the surface it didn't seem like a tough problem to solve, and yet, not a single member of the best team I've ever worked on could explain the entire problem. In short, it wasn't just complicated, it was deceptively complicated - which made matters worse. The problem was large, and we attempted to tackle it (clearly, not intentionally) by building a team that was too large.

After a year of practicing Collective Code Ownership I found that most areas of the codebase would have a primary person who deeply understood it's purpose, and almost everyone else knew very, very little about it. Those with shallow knowledge could successfully edit a method, but a change of greater size would require pairing with the primary person. The problem was large, and we were spread too thin.

The application was already split into logical components, and my solution was to move to a Weak Code Ownership model - with the requirement that each component would have at least 3 "owners" that deeply understood each component. You would still be required to know what each component did, but you wouldn't ever be required to even look at the code for that component. I believe this solution would have solved our depth issue, but so would breaking up our large team.

If your team is so large that working in every area of the codebase results in shallow knowledge of over 40% of your codebase, that might be a sign that you could benefit from splitting the team into smaller groups with more specific responsibilities.

And, just in case you are wondering, pairing isn't the solution to having a team that is too big.

I'd like to thank the following people for reviewing and providing feedback on this entry: J. B. Rainsberger, Peter Royal, Nick Drew, Mike Rettig, John Hume, Clinton Begin, Pat Kua, Nat Pryce, Patrick Farley, & Dan North.

organizing a speakerconf

About 5 years ago I expressed an idea to Josh Graham.

Let's take the parts of traditional conferences that the presenters like the most, and make that an entire conference. Specifically, we should have plenty of discussion time with drinks available & we should have group dinners every night.

I'd expressed this idea to a few different people, and never done anything with it. Luckily, by the time I'd woken up the following morning, Josh had already set up the speakerconf website. Organizing speakerconf has been an interesting and enlightening journey. The past few blog posts have been about what speakerconf is, what presenters are responsible for, & logistics - this post is about how we got started and the organization and participation lessons I learned.

The Beginning
Once Josh created the website I knew it was time to see if we could actually make the event happen. I started by emailing my network to see who liked the idea and would attend. I always knew that 15 was the minimum number of presenters that I would want for a speakerconf, so I sent the emails and waited to see if we had enough interest or not - luckily we did. I also knew I wanted to have a group dinner every night, and the only way to convince 15 people to attend the same restaurant was to pay the bill - so I set out to find sponsors. The management at DRW is incredibly supportive and they quickly offered to sponsor. I also had a great relationship with Forward and they immediately offered to sponsor as well. With both of those sponsors committed, we had the minimum amount of money required to pay for the conference room and cover the cost of the group dinners. At that point there was no looking back, and we began building the hype via Twitter.

Quality, not Quantity
I believed that getting a commitment from a few high-profile speakers would cause plenty of other speakers to jump on board, but the speakerconf idea didn't really generate much buzz the first year (from my point of view). Once the idea became a reality, things changed drastically. The first event was a huge success. We had over 30 people sign up for speakerconf 2010 (in those days, if anyone you knew was going you could go as well). The second event was also a success, but we all agreed that 30+ was too many people. Following 2010, speakerconf became an invite only event and was limited to 25 people.

We were happy with 25 people at each speakerconf, but we ended up learning that less is an even better situation. speakerconf Aruba 2012 had 12 people cancel, which left us in a terrible place. Replacing a speakerconf presenter is extremely hard. The net result was that only 18 people attended, and it worked out perfectly. The collaboration, quality, and enjoyment all seemed to go up significantly. I believe it became a case of addition by subtraction, due to the interactions becoming much richer. After the success of Aruba 2012 Josh and I decided that future speakerconfs will have between 18-20 presenters.

Another Point of View
At speakerconf, your participation in discussion is just as important as your presentation. At the first few speakerconfs I had unreal expectations about people spending every waking minute participating. It was the center of attention for Josh and I, but for the presenters it was just a great event that they needed to responsibly balance with their other commitments. After the first 2 speakerconfs I learned that we needed to set aside time for everyone to take care of work and personal business. At this point, Josh and I allocate 2 hours before dinner each night where presenters can take care of non-speakerconf related issues.

You do not talk about speakerconf
As programmers, we like to optimize. Sadly, I've wasted countless hours and other people's time at speakerconf by discussing how to tweak speakerconf. My intentions were good, I wanted to improve the event. Unfortunately, there's two issues with this: We could be talking about technology instead, and it brings attention to any issues I'm noticing. I believe the old advice that the party takes on the mood of the host - and I don't want to bring speakerconf presenters into my perfectionist world that constantly focuses on otherwise unseen flaws. Plus, if I'm hanging out with some of the best from our industry, it's much more beneficial to hear about what they're working on these days.

I'm not going to lie, it's not easy to stop myself for asking for feedback while the event is running, but I do believe it's the right choice. I'm more than happy to discuss tweaks if someone else brings it up, and I definitely solicit feedback once the event is over; however, speakerconf time is for discussing technology, and the conference benefits when I remember that simple suggestion.

The Best, Last
The last 2 lessons are unquestionably the most important: be flexible and get a co-founder. Brian Goetz mentioned to me many times - get smart people together and let them figure out how to have a good time. Whenever people suggest we tweak something, Josh and I do our best to accomodate and see where it takes us. Above all else, we remove distractions and get out of the way, so presentations or ideas can reach their full potential. Occasionally that means letting a presentation run long, switching the presentation order, taking a quick break, or moving to open discussion ahead of schedule. The schedule always suffers, but the experience always benefits. At this point, Josh and I have learned to roll with whatever comes our way, as long as it enhances speakerconf.

speakerconf requires Josh. I hear it's true that start-ups need a co-founder, and I couldn't have done speakerconf without Josh. I tell him I'm quitting at least once a year. Josh seemed like he might quit last year. However, we carry each other when we need to, and I'm very proud of what we put out each year.

trust at speakerconf

Trust plays a very important role at speakerconf. speakerconf is known as a place where you can be honest without fear of repercussions. Every year we have several presentations that are confidential, and those are often some of the most interesting presentations. Josh and I have made several decisions that encourage a high level of trust, and this blog entry describes those choices.

... it's who you know
Josh and I don't tend to keep people we can't trust within our networks. Therefore, when we began inviting people to speakerconf we already knew we were assembling a group of people that could be honest with each other. I believe the speakers had mutual trust with Josh and I, and that trust was easily extended to other people that Josh and I trusted.

After the first year we started inviting people using a simple rule: 75% of all invites will go to alumni. speakerconf alumni will already have existing relationships with other alumni, and they'll be more willing to share - due to their previous experiences at speakerconf. We always want new people and fresh ideas at speakerconf, but we don't want to lower the trust level by throwing in a larger group of people who haven't yet built strong relationships. We've experimented with inviting less alumni and never been happy with the results. For speakerconf 75% seems to be the magic number.

These days, Josh and I only invite new people to speakerconf based on alumni recommendations. Alumni recommendations are often for people from various backgrounds; which is perfect, as Josh and I prefer to have presenters with diverse experiences. I believe the key to embracing diversity while not compromising on trust is to focus on presenters that have mutual respect for each other despite their backgrounds.

Encourage relationship building
Every speakerconf begins with a meet & greet networking session. This session is usually at a hotel lounge area, and it's only purpose is for each of the speakers to get a chance to meet each other. Following the meet & greet, everyone is invited to an "unoffical" dinner. It's not required or covered by the conference, but it allows people to continue to get to know each other, if they like. Very rarely do any presenters skip this opening dinner.

During both the dinners and the open discussions we provide conference sponsored drinks - to get the honest discussions flowing. I'll be honest, when I'm stone cold sober I can be a bit shy. I get the impression that other speakers sometimes share this personality trait. The open discussions do often begin a bit timidly; however, after a round or two we quickly move to pulling no punches and diving head first into whatever interests us the most.

Off the record
At speakerconf, we don't even provide the option to record your talks. It would be great if we had some material that we could put on line for the rest of the community to benefit from, but we're not willing to sacrifice any level of honesty for the sake of information sharing. We've surveyed the presenters and they've unanimously agreed that they do not want their presentations to be recorded, and they would likely change what they said simply if a camera were present in the meeting room.

It's obvious, but we also remind everyone that if their presentation contains any sensitive information then they should begin by asking the audience not to repeat anything from the presentation. Also, at the beginning of every speakerconf we ask all the speakers to refrain from repeating (e.g. tweeting) any information without first consulting the source of the information.

Network
We don't wear nametags at speakerconf, and we don't go around the room and introduce ourselves. I personally hate nametags, and we found the introduction-go-around to be a bit dry and awkward. Instead, before every presenter speaks, Josh stands up and provides several facts about a person, and a single lie. It's up to the audience to decide which statements are which, though it's generally obvious due to the lie being the statement that caused everyone to laugh. It's fun, and turns the traditional awkward introduction into something you look forward to at the beginning of each presentation.

We don't officially organize seating for lunch or dinner at speakerconf, but we do encourage people to rotate who they sit with while they're dining. We found that the level of trust goes up greatly after you've taken the time to break bread with someone, and we want to encourage that trust building in anyway possible. Like I said, we don't organize anything specific here, we just lead by example; we try to pick our company for our next meal based on who we've spent the least amount of time with.

That's about all we do. There's nothing revolutionary about our methods; however, the result is an environment where trust goes a long way towards helping a presentation reach it's full potential.

So, You Dropped Out of College

I was recently chatting to some friends about all the apprenticeship experiments going on and (name withheld to protect the innocent) lamented:

So, what happens when they quit that first job (or worse, get laid-off) and their only skill is writing a Rails app?

I know what happens. It happened to me.

Okay, I didn't actually participate in an apprenticeship program that taught me Rails. However, I did drop out of college after gaining some experience programming, get a job building web applications, and I was laid off about a year later. Getting another job wasn't so hard, but it was hard to find a job that paid as well. It turns out, there's not a huge demand for people who don't have formal education or much experience, go figure.

While I was looking for a new job I noticed that there were significantly more jobs for people with certain skills. I'm sure it differs by location and time - in Jacksonville FL, around 2000, 75% of the jobs openings were looking for people with Microsoft .net skills. Sadly, I did not have .net skills. That was when I learned that I needed to have not just the skills necessary to do my job today, but also the skills necessary to land a new job tomorrow. Since I wasn't already in that situation, I had to settle for a job making .5 of my previous salary; however, my new job did allow me to learn .net.

I firmly believe: if you have less than 4 years of experience, you should always know what technology is in the highest demand for your area, and you should be proficient enough that you can do well in an interview. Perhaps your day to day job already teaches you the most highly sought after skills, fantastic, you should probably learn the 2nd most sought after skill. Don't wait until you're already looking for a job to try to learn a new technology. The truth is, learning something new will almost certainly help you with your current job as well, so everyone wins.

note: searching hipster-programming-jobs.com to find out what's in demand is a selection bias failure. Use monster.com & dice.com. The goal is to appeal to as many companies as possible, not just the cool ones. You can focus on your dream job once you've got enough experience to ensure that your career doesn't suffer a failure to launch.

Whatever technology you choose will certainly have specific books that you should read; however, there are other books that will also help make up for the experience you (sadly, we) missed by not getting a Computer Science degree.

Refactoring was the first book I ever read that really opened my eyes to better ways to program. It's a pretty easy book to read, even for a beginner, and I can't recommend it enough. It's likely that you find that the most in-demand technology is Java or C#, and Refactoring will complement learning either of those languages. However, if you prefer, there's a Refactoring: Ruby Edition as well.

Once you've read refactoring, it's good to dig into Patterns of Enterprise Application Architecture (PofEAA). PofEAA contains patterns that are found all over the enterprise world (including ActiveRecord). You should read this book not because it is a bible, but because it shows you various solutions to commonly encountered problems. It gives you ideas about how other people are solving problems, and provides reference data that you can refer to if you encounter any of those issues in the future.

Those two books will demonstrate to potential employers that you're interested in building successful business applications; however, you'll also want to read a few books that diversify your experience and open your eyes to what can be done by using less mainstream techniques.

The wizard book and the dragon book sound like interesting fictional books, but are actually building blocks for your programming career. Both books are widely regarded as classics that any serious programmer should read. That said, they are not easy to digest. I'm not going to claim it will be fun, but you should push your way through both of these books - and then do it again a year later. These books can help shape your career in a positive way, especially the wizard book, perhaps more than any other text available.

Once you've pushed through both of those books, there are a few follow up options. If you preferred the wizard book, I would recommend following up with The Joy of Clojure or Learn You a Haskell For Great Good. You're unlikely to get a job using either technology, but both will broaden your horizons and also demonstrate to potential employers that you are deeply interested in increasing your programming capabilities. If you preferred the dragon book, I would move on to Domain Specific Languages. Domain Specific Languages is the definitive guide to DSL programming, and provides you with many examples of how to put DSLs to practical use within your applications.

Once you've made it through all (or most) of those books, you shouldn't have any problem finding a job that is both satisfying and also pays well.

scheduling, travelling to, and dining at speakerconf

Convincing 20 industry leaders to set aside 4-5 weekdays to come to a conference is not an easy thing, especially when said conference is also very hard to describe to the person (or people) who are approving the time off, travel, & expenses. With this in mind, Josh and I designed speakerconf to appeal to programmers who work very hard, and are often on the road.

Weekdays. I strongly hold the opinion that conferences should not be on weekends [more info]. Most of the people who attend speakerconf work very hard on an average week, and I don't want take a weekend away from them, their friends, or their family. I'm a very firm believer in work / life balance, and I don't want to be part of tipping the scales even farther in the 'work' direction. Additionally, it makes sense as a conference organizer. I want people showing up to speakerconf rested, refreshed, and ready for several days of intense collaboration. It's not enough to give a great presentation at speakerconf, you need to be on your game for 10-12 hours a day and you need to be able to do it for 3-5 straight days.

My opinion is not universally held - some people can't or don't want to be away from the office for that many days. While I see their point of view and note the benefits of switching to starting on Sunday, I simply don't believe that the trade-offs are a net win for speakerconf. Therefore, the best thing that Josh and I can do is ensure that speakerconf is such a "don't miss" event that people believe it is unquestionably worth being out of the office for a week.

Select an easily accessible location. Easily accessible has always been a requirement of mine for speakerconf. Like I mentioned, speakerconf presenters are often on the road - the last thing I want to do is require an additional connection or an extended drive. Aruba is a direct flight from many cities in the US, and the speakerconf hotel is a 15 minute drive from the airport. Aruba was great for the first few speakerconf events; however, when Josh and I created speakerconf we never anticipated that we would draw so much interest from people in Europe as well. Sticking with our ideals about easy accessibility, speakerconf now rotates between a location that is a direct flight from most US airport hubs and a location that is a direct flight from most EU airports.

Select an isolated location. Running a speakerconf in Rome really drove this point home - last February I asked if 2013 should be in Austin, SF, NYC, or Aruba. The majority preferred Aruba, and, more importantly, those that preferred Aruba were also the ones who were most excited about returning in 2013. Many presenters pointed out that if we were in an actual city they would be more likely to be distracted. The final dinner in Aruba proved their point. In Rome in 2011, 1/3 of the presenters went out to dinner* on the final night. Conversely, in Aruba 17/18 presenters went to dinner and all 17 continued their discussions at the hotel bar for many hours following dinner. The isolated location provided fewer distractions, and seemed to facilitate much deeper interactions.

Select a location with a concentration of restaurants. speakerconf takes a very different approach to dining. Josh and I aim to not only host an amazingly educational event, but to also provide the best dining experience of any conference. We don't provide conference center food for breakfast or lunch, instead we select hotels that are near great local restaurants that serve reasonably priced breakfast and lunch options. Dinners are done at some of the highest rated local restaurants - and we require the ability to select from several different options.

Dine well. Josh and I love food & wine, and our preferences definitely show in our dinner selections. The conference sponsored dinners at speakerconf are generally at restaurants that I prefer to take my wife to when we visit those cities on vacation. The dinner wine selections are usually done based on what Josh has loved drinking in the past. As an example, the conference dinners in Munich are at Boettners & Vue Maximilian, the number 1 & number 3 rated restaurants in Munich according to zagat.com. I could go on and on about the type meals that we aim to provide at speakerconf, but sometimes a picture truly is worth a thousand words - here are the 1,000 that describe a dinner at speakerconf.

Group dinners are better if you split to tables of 3-5. Many people loved being part of a large, group dinner table, but the conversations did suffer a bit. In 2011 we began making dinner reservations for "a party of 20, but we would like 5 tables of 4", and the conversations became longer, deeper, and more productive. As a side benefit, the restaurant staff seems to find this easier to manage as well - which means less issues with 19 people having their food, and awkwardly waiting for the 1 meal that didn't come out right.

On the surface it may look like speakerconf is a week off work, in a vacation location, dining like a king; however, if you look closely, each of those choices is based on a practical choice. The scheduling is a net win for everyone involved. The location facilitates distraction free, deep collaboration. The dining enhances the event by allowing the attendees to focus on collaboration while Josh and I worry about location selection, providing food that meets all dietary restrictions, and ensuring that the cost of dinner doesn't deter attendance. The result (or sum) is also greater than the parts - we provide an easily accessible, deeply educational, & enjoyable experience, which encourages some of the best of the industry leaders to attend, who's attendance encourages more of the industries leaders to attend.

* The final dinner is neither required, nor sponsored - thus it's completely fine for people to opt-out.

Clojure: name function

The 'name' function is a clojure function that returns the string for a keyword, symbol, or string.

name - function
Usage: (name x)
Returns the name String of a string, symbol or keyword.

At first glace this might not seem that interesting; however, it's good to know 'name' if you've ever been surprised by (str :foo) => ":foo". If you have a ruby background (as I do), you probably expected the result to be "foo", spent a bit of time looking, and found that (name :foo) was actually what you were looking for.

That's helpful, but not particularly exciting. Perhaps a more interesting application of name is the ability to normalize all keys as strings and destructure. For example, say you're designing a library that monitors threads and you want to be able to pass in warning and error thresholds. Usage of your functions may look like the following examples

(monitored-threads/create :warn-threshold 100 :error-threshold 200)
(monitored-threads/create "warn-threshold" 100 "error-threshold" 200)

Assuming a simple function that updates keys:

(defn update-keys [m f]
  (reduce (fn [r [k v]] (assoc r (f k) v)) {} m))

You can now write your create function as:

(defn create [& {:as m}]
  (let [{:strs [warn-threshold error-threshold]} (update-keys m name)]
    ; do work, son
    ))

speaking at speakerconf

Speaking at speakerconf is nothing like speaking at a traditional conference. It took us a few years to tweak our ideas around presentations - and this blog post is about what we've come to consider 'the speakerconf way' (with respect to speaking)

There are no abstracts or pre-announced talks. From the beginning Josh and I have believed that we want people to be able to speak about whatever is most interesting to them. Too many times in my career I have had a talk accepted to a conference, and by the time the conference comes around I'm on to something new. I honor my commitment and give a talk on the accepted abstract, but I always feel a bit guilty for presenting information that's already somewhat dated. speakerconf completely avoids this issue by neither requesting nor accepting abstracts. In fact, many presenters prepare a few different presentations and just-in-time choose whichever presentation they believe will be better received.

Prepare only 10 minutes of content. speakerconf began with 20 minute time-slots exclusively. Over the following few years we toyed with 5, 15, & 30 minute talks. In general, the 5 minute talks turned out very well, the 15 minute talks turned out well most of the time, and the 30 minute talks always seemed to stretch far too long. Each year Dave Thomas recommended that we give each presenter 10 minutes. We finally made the 10 minutes of content rule change in 2011 and have never looked back. 10 minute talks are perfect for speakerconf. If people are into a topic then the questions end up stretching the session out to 30 (deeply engaged) minutes, if people aren't into a topic then you're off stage before people get bored. Since we've made this rule change, people have been generally happy with the presentations as a whole, and we've yet to have a presenter give a presentation that wasn't enjoyed by the majority of the audience.

Presentations go on as long as they have to. John Hughes inspired this one. Like I mentioned in the last paragraph, we give everyone 10 minutes for content, but let the audience drive the actual length of the presentation with their questions. The speakerconf audiences are inquisitive, so we still need to put an upper bound of 30 minutes on a talk. Still, most talks manage to create plenty of deep discussion in their 30 minute windows.

Presentations first, unstructured conversation second. A few years ago we switched things up by splitting up the presentations, and it didn't work well - people couldn't get back in the mood for presentations. Now, each day starts with presentations that spark ideas and then goes to unstructured conversation about those ideas.

Alumni speak first. Originally, we allowed speakers to request their speaking slots. In the 2nd year Matt Deiters requested and spoke in the 2nd speaking spot, and quickly regretted it. Speaking at speakerconf requires a bit of calibration. The audience at speakerconf isn't like the vast majority of conferences, and you do need to alter your presentation style a bit - go faster, take out filler or elementary slides, expect frequent interruptions. After a few days of speakerconf it's easy to fall in the groove; however, Matt didn't get that luxury. Based on his suggestion, each speakerconf since then has scheduled all alumni ahead of new-comer presentations.

Dynamic speaking order. Each speakerconf does have a suggested schedule; however, we also offer the opportunity to 'get next' at any time. At speakerconf Rome 2011, Francesco Cesarini noticed that his presentation would go very well if it followed Scott Farquhar's. Unbeknownst to the presenters and organizers, some people end up presenting on very similar ideas. Once the audience gets into a subject, it makes sense to allow presentations with complementary ideas topics to be grouped together. Therefore, at speakerconf any presenter that believes their presentation will nicely expand on the current presentation can simply let the organizers know that they've 'got next' and they'll be moved into the next speaking time-slot.

Everyone speaks. For the first few years we had attendees and presenters; however, we found that there were a few issues with non-speaking audience members. First of all, when you speak everyone knows a bit about who you are and what you do; conversely, the non-speaking attendees were always a mystery. Additionally, speaking audience members always participated significantly more, as they had given people a topic to approach them about. Lastly, we often had non-speaking attendees who the other speakers actually wanted to hear from. In the end it just didn't make sense to invite very talented people who were only participating in the open discussions. Now, and in the future, there are no attendees, if you attend speakerconf, you present.

The speakerconf way is very nimble. We don't know what people are going to talk about, how long they're going to talk, or when they're going to give their presentation. In fact, the only thing we do know is that you are eventually going to speak about something. Obviously this isn't something that every conference could adopt, but we've found that it greatly enriches the speakerconf experience. The speakerconf way provides a quick exit for anyone who's idea isn't going over quite as smoothly as they'd anticipated, but, more importantly, it allows the good presentations to reach their full potential.

a typical day at speakerconf

At this point, speakerconf has run 5 times in 4 years - in Aruba and Italy. Josh and I are happy with what we've created, and a few follow up entires will list the attributes of speakerconf that make it a unique and successful event. This post, however, should serve as a quick view into what a typical day is like at speakerconf.

the morning
Each day begins unofficially at breakfast. There's no assigned meeting time, but people generally start appearing around 9:00am. There are no assigned tables, and people usually just break off into small, ad hoc groups of 3 or 4. It's rare to find one of these tables not talking about what interested them from the day before, or what they're looking forward to discussing that day. Following breakfast, people begin filling into the conference room, and presentations begin at 10:00am sharp. Each presentation generally takes between 10 and 30 minutes. There's usually a ton of questions that go with each presentation, thus the variable talk time-slot. We attempt to get 4 presentations done each morning, and usually end up leaving 'late' for lunch.

the afternoon
Lunch generally lasts an hour, and is done at one of the local restaurants (not in the conference room). Attendees break into small groups and go to different restaurants. Most often, people go to lunch with someone they want to discuss a specific idea with, or someone they haven't previously dined with. After lunch everyone makes their way back to the conference room for 4 more presentations. On a good day, we'll be done with presentations by 3pm; however, (due to the q&a for each presentation) we're usually done with presentations between 3:30pm and 4:00pm.

Once the presentations end we move out of the conference room and into a space that allows us to have many smaller discussions. The split is ad hoc once again, and, just like lunch, is often based on digging further into a presented topic or meeting a new person. These discussions often begin with most of the presenters in attendance, and people only seem to leave if they have personal or business matters to attend to. These smaller discussions end (or rather, are put on hold) at 6:45pm at the latest.

the evening
Dinner is always at a local restaurant (outside of the hotel), and usually begins around 7:00pm. Just like every other meal, attendees break up into unassigned, smaller groups of 3-6 people. Dinners are usually at least 3 courses, and offer plenty of time to dig deep into whatever subject drove you to select your dinner companions. Once dinner is complete everyone heads back to the hotel, and those that still have the energy to have further discussions generally make their way to the hotel bar. It's not uncommon for those that have already completed their presentations to stay out past 2am - and still get themselves to the 9:00am breakfast the following morning.

That's a pretty typical day at speakerconf. If it doesn't sound exhausting, then I haven't done a very good job of describing it. However, it's equally exhausting as it is inspiring, and the days at speakerconf are some of my favorite days of the year.

Clojure: expectations & with-redefs

In general, when I'm writing tests, the pure functions end up as bare expects and the impure functions end up as scenarios. The following contrived namespace keeps a list of users and allows you to get the full name of each user.

The tests for this namespace would often look something like the following code:

It feels natural to put the with-redefs in a scenario, since scenarios also support (and often make use of) stubbing, localize-state, & freeze-time. However, there's really no reason that you need to use a scenario if you're simply looking to redef a var.

The following test provides the same level of functionality verification, without needing to use expectations.scenarios:

scenarios are great, but these days I try to keep things simple with bare expectations whenever possible.