Emacs Lisp: Toggle Between a Clojure String and Keyword

When I was doing a fair bit of Ruby I often used the TextMate's shortcut (Ctrl+:) to convert a Ruby String to a Symbol or a Ruby Symbol to a String. It's something I've periodically missed while doing Clojure, and yesterday I found myself in the middle of a refactoring that was going to force the conversion of 5+ Clojure Keywords to Strings.

The following emacs lisp is my solution for toggling between Clojure Strings and Keywords. The standard disclaimers apply - it works on my machine, and I've never claimed to know emacs lisp well.

A quick video of the behavior:

So, You Dropped Out of College

I was recently chatting to some friends about all the apprenticeship experiments going on and (name withheld to protect the innocent) lamented:

So, what happens when they quit that first job (or worse, get laid-off) and their only skill is writing a Rails app?

I know what happens. It happened to me.

Okay, I didn't actually participate in an apprenticeship program that taught me Rails. However, I did drop out of college after gaining some experience programming, get a job building web applications, and I was laid off about a year later. Getting another job wasn't so hard, but it was hard to find a job that paid as well. It turns out, there's not a huge demand for people who don't have formal education or much experience, go figure.

While I was looking for a new job I noticed that there were significantly more jobs for people with certain skills. I'm sure it differs by location and time - in Jacksonville FL, around 2000, 75% of the jobs openings were looking for people with Microsoft .net skills. Sadly, I did not have .net skills. That was when I learned that I needed to have not just the skills necessary to do my job today, but also the skills necessary to land a new job tomorrow. Since I wasn't already in that situation, I had to settle for a job making .5 of my previous salary; however, my new job did allow me to learn .net.

I firmly believe: if you have less than 4 years of experience, you should always know what technology is in the highest demand for your area, and you should be proficient enough that you can do well in an interview. Perhaps your day to day job already teaches you the most highly sought after skills, fantastic, you should probably learn the 2nd most sought after skill. Don't wait until you're already looking for a job to try to learn a new technology. The truth is, learning something new will almost certainly help you with your current job as well, so everyone wins.

note: searching hipster-programming-jobs.com to find out what's in demand is a selection bias failure. Use monster.com & dice.com. The goal is to appeal to as many companies as possible, not just the cool ones. You can focus on your dream job once you've got enough experience to ensure that your career doesn't suffer a failure to launch.

Whatever technology you choose will certainly have specific books that you should read; however, there are other books that will also help make up for the experience you (sadly, we) missed by not getting a Computer Science degree.

Refactoring was the first book I ever read that really opened my eyes to better ways to program. It's a pretty easy book to read, even for a beginner, and I can't recommend it enough. It's likely that you find that the most in-demand technology is Java or C#, and Refactoring will complement learning either of those languages. However, if you prefer, there's a Refactoring: Ruby Edition as well.

Once you've read refactoring, it's good to dig into Patterns of Enterprise Application Architecture (PofEAA). PofEAA contains patterns that are found all over the enterprise world (including ActiveRecord). You should read this book not because it is a bible, but because it shows you various solutions to commonly encountered problems. It gives you ideas about how other people are solving problems, and provides reference data that you can refer to if you encounter any of those issues in the future.

Those two books will demonstrate to potential employers that you're interested in building successful business applications; however, you'll also want to read a few books that diversify your experience and open your eyes to what can be done by using less mainstream techniques.

The wizard book and the dragon book sound like interesting fictional books, but are actually building blocks for your programming career. Both books are widely regarded as classics that any serious programmer should read. That said, they are not easy to digest. I'm not going to claim it will be fun, but you should push your way through both of these books - and then do it again a year later. These books can help shape your career in a positive way, especially the wizard book, perhaps more than any other text available.

Once you've pushed through both of those books, there are a few follow up options. If you preferred the wizard book, I would recommend following up with The Joy of Clojure or Learn You a Haskell For Great Good. You're unlikely to get a job using either technology, but both will broaden your horizons and also demonstrate to potential employers that you are deeply interested in increasing your programming capabilities. If you preferred the dragon book, I would move on to Domain Specific Languages. Domain Specific Languages is the definitive guide to DSL programming, and provides you with many examples of how to put DSLs to practical use within your applications.

Once you've made it through all (or most) of those books, you shouldn't have any problem finding a job that is both satisfying and also pays well.

sidenum function

The sidenum function appends a sign to a number based on a side. The name sidenum was chosen due to it's similarities with signum in spelling and behavior (dealing with signs).

Definition

---

The sidenum function of a side x and a number y is defined as follows:

note: 'bid' can be replaced with 'buy', and 'ask' can be replaced with 'sell' or 'offer'. The string representation or type of side varies by trading system.

Usage

---

The sidenum function appears in most trading applications, though not always named 'sidenum'. It can be helpful for both position related math and conserving screen real-estate by displaying 2 concepts (side & quantity) as 1 (a signed quantity).

Examples

---

Java

    public static int sidenum(String side, int quantity) {
        if (side.equals("ask"))
            return -quantity;
        return quantity;
    }

Clojure

    (defn sidenum [side quantity]
      (if (= side "ask")
        (- quantity)
        quantity))

Ruby

    def sidenum(side, quantity)
      if side == "ask"
        -quantity
      else
        quantity
      end
    end

Sacrificing some IDE capabilities

When discussing adopting Clojure or JRuby there are often heated discussions concerning the pros and cons of working with a language that has less IDE support. This blog entry will focus on the common concerns I hear.

I've never met anyone who has mastered Ruby or Clojure and had the opinion that they are more productive in Java. I think that's the best "proof" that the pros outweigh the cons.*

Other than that, things get complicated. The problem is, I understand where reluctant adopters are coming from. But, I just don't see any way to convince them to make the leap.

Many Java programmers do a bit of Ruby or Clojure and make a determination. I think their concerns are inflated by 3 factors: api/library fear, shallow understanding of powerful language features, and inability to optimize problem resolution.

A common concern is that learning a new language requires learning new libraries. Even though Java's libraries are available in Clojure or JRuby, both languages still have their own apis/libs to learn. In Java, you often use auto-completion. In Ruby and Clojure some auto-completion is available; however, it's often suspect. Few people use a Java REPL, so they don't understand how helpful/powerful a REPL is. In Clojure and Ruby, the REPL is often the most effective way to learn new apis/tools. Since most Java developers haven't experienced the value of the REPL, they exaggerate the overhead of learning apis/libs.

Writing idiomatic Java using Ruby or Clojure will produce painful and likely unmaintainable code. Writing idiomatic Ruby and Clojure often require understanding the powerful aspects of a language (metaprogramming, method_missing, macros, duck-typing). Understanding those aspects of a language can greatly change the design of a system. The result of mastering language features is often concise and significantly more maintainable code. However, if you aren't able to see the concise/maintainable version, you will picture a design that may likely be hard to manipulate without IDE support. Enter unnecessary concern and doubt.

Lastly, when things go wrong you need to know how to fix them. Fixing issues is often a mix of tweaking the language and mixing in a few tools. In Java it's often Ignore the noise in the stacktrace, Click a few links and open a few files, Drop in some break points or print lines or change a few tests. Getting to the source of the problem efficiently requires an IDE. In Clojure and Ruby I have those tools (since I do my Clojure and Ruby in IntelliJ), but I also have the REPLs. I know what noise to ignore. Even without those tools (when I used TextMate for Ruby), I know what patterns generally represent what issues. I can see the signal in the noise, and I have an additional tool (the REPL) to help me determine what is signal and what is noise.

However, if you don't know what is noise and what is signal, and (once again) you don't understand the value of a REPL then you exaggerate the cost of problem resolution.

Cosmin Stejerean recently mentioned "To me good IDE support for Clojure is very much like using training wheels on a bicycle."

That's almost true. I'll freely admit that I'd love some free refactoring. It would make me more productive. However, I'm already more productive so it's icing on the cake, not a reason to stop me from using the language.

Unfortunately, those training wheels are very much a deal-breaker to conservative adopters.

* I have recently heard of a few projects starting as Python and switching to Java, but I haven't ever done any Python, so I wont speculate as to why that happened.

Refactoring: Ruby Edition available.

Refactoring: Ruby Edition is available (and In Stock) on amazon.com.



Sorry it took so long, I hope it is worth the wait.

Refactoring: Ruby Edition available on Safari

Refactoring: Ruby Edition is now available on Safari as a Rough Cut.

Ruby: Recording Method Calls and Playback With Inject

Sometimes you want to call methods on an object, but you want to delay the actual execution of those methods till a later time.

For example, in expectations you create a mock at parse time, but you actually want the mock to be available at execution time.

class SystemProcess
  def start
    puts "started"
    new StartedProcess
  end
end

Expectations do
  expect SystemProcess.new.to.receive(:start) do |process|
    process.start
  end
end

In the above code you define what you expect when the file is parsed, but you actually want the process expectation to be set when the do block is executed. This can be (and is) achieved by using a recorder to record all the method calls on the process object. At execution time the method calls are played back and the initialized process object is yielded to the block.

The code for a recorder is actually quite trivial in Ruby.

class Recorder
  attr_reader :subject
  def initialize(subject)
    @subject = subject
  end
  
  def replay
    method_stack.inject(subject) { |result, element| result.send element.first, *element.last }
  end
    
  def method_stack
    @method_stack ||= []
  end
  
  def method_missing(sym, *args)
    method_stack << [sym, args]
    self
  end
end

Here's an example usage of a recorder.

class SystemProcess
  def start(in_seconds)
    puts "starting in #{in_seconds}"
    sleep in_seconds
    StartedProcess.new
  end
end

class StartedProcess
  def pause(in_seconds)
    puts "pausing in #{in_seconds}"
    sleep in_seconds
    PausedProcess.new
  end
end

class PausedProcess
  def stop(in_seconds)
    puts "stopping in #{in_seconds}"
    sleep in_seconds
    self
  end
end

Recorder.new(SystemProcess.new).start(1).pause(2).stop(3).replay
# >> starting in 1
# >> pausing in 2
# >> stopping in 3

The only thing worth noting is that by using inject you can use a method chain that returns different objects. Traditional versions of a recorder that I've seen often assume that all the methods should be called on the subject. I prefer the version that allows for object creation within the fluent interface. In practice, that's exactly what was needed for recording and playing back Mocha's expectation setting methods.

Refactoring: Ruby Edition available on Amazon

It's been about 18 months since my first entry about Refactoring: Ruby Edition. It's been a long journey, but the book should be seeing day light in the next few months.

It's available for pre-order on Amazon now, and it should also be available on Safari in the near future.

Thanks to everyone who helped, and I hope the book is as fun to read as it was to write.

Ruby: Move a Method from a Class to a Module Definition

I was recently working with a framework that reopened and defined a method on Object. I wanted the behavior of the framework, but I also wanted to define my own behavior. This is generally the case for alias_method_chain or one of the other Alternatives for Redefining Methods, but my circumstances (to be discussed in a subsequent post) prevented me from using one of the better known solutions.

The solution that worked for me was to move the original method definition to a module.

class Speaker
  def say_hello
    "hello"
  end
end

module EnglishSpeaker
  expects_method = Speaker.instance_method(:say_hello)
  define_method :say_hello do |*args|
    expects_method.bind(self).call(*args)
  end
end

class Speaker
  undef say_hello
end

Speaker.new.say_hello # => -:18: undefined method `say_hello' for #<Speaker:0x285b4> (NoMethodError)
Speaker.new.extend(EnglishSpeaker).say_hello # => "hello"

You may never need this technique, I only needed it once in the past 2.5 years. But, when it applied I found it to be significantly better than the alternatives.

Ruby: inject

I love inject. To be more specific, I love Enumerable#inject. I find it easy to read and easy to use. It's powerful and it lets me be more concise. Enumerable#inject is a good thing.

Of course, I didn't always love it. When I was new to Ruby I didn't understand what it was, so I found it hard to follow. However, finding it hard to understand didn't make me run from it, instead I wanted to know what all the hype was about. Enumerable#inject is an often used method by many great Rubyists, and I wanted to know what I was missing.

So, to learn about Enumerable#inject I did what I always do, I used it every possible way I could think of.

Example 1: Summing numbers
Summing numbers is the most common example for using inject. You have an array of numbers and you want the sum of those numbers.

[1, 2, 3, 4].inject(0) { |result, element| result + element } # => 10

If the example isn't straightforward, don't worry, we're going to break it down. The inject method takes an argument and a block. The block will be executed once for each element contained in the object that inject was called on ([1,2,3,4] in our example). The argument passed to inject will be yielded as the first argument to the block, the first time it's executed. The second argument yielded to the block will be the first element of the object that we called inject on.

So, the block will be executed 4 times, once for every element of our array ([1,2,3,4]). The first time the block executes the result argument will have a value of 0 (the value we passed as an argument to inject) and the element argument will have a value of 1 (the first element in our array).

You can do anything you want within the block, but the return value of the block is very important. The return value of the block will be yielded as the result argument the next time the block is executed.

In our example we add the result, 0, to the element, 1. Therefore, the return value of the block will be 0 + 1, or 1. This will result in 1 being yielded as the result argument the second time the block is executed.

The second time the block is executed the result of the previous block execution, 1, will be yielded as the result, and the second element of the array will be yielded as the element. Again the result, 1, and the element, 2 will be added together, resulting in the return value of the block being 3.

The third time the block is executed the result of the second block execution, 3, is yielded as the result argument and the third element of the array, 3, will be yielded as the element argument. Again, the result and the element will be added, and the return value of the block for the third execution will be 6.

The fourth time will be the final time the block is executed since there are only 4 elements in our array. The result value will be 6, the result from the third execution of the block, and the element will be 4, the fourth element of the array. The block will execute, adding four plus six, and the return value of the block will be 10. On the final execution of the block the return value is used as the return value of the inject method; therefore, as the example shows, the result of executing the code above is 10.

That's the very long version of how inject works, but you could actually shortcut one of the block executions by not passing an argument to inject.

[1, 2, 3, 4].inject { |result, element| result + element } # => 10

As the example shows, the argument to inject is actually optional. If a default value is not passed in as an argument the first time the block executes the first argument (result from our example) will be set to the first element of the enumerable (1 from our example) and the second argument (element from our example) will be set to the second element of the enumerable (2 from our example).

In this case the block will only need to be executed 3 times, since the first execution will yield both the first and the second element. The first time the block executes it will add the result, 1, to the element, 2, and return a value of 3. The second time the block executes the result will be 3 and the element will also be 3. All additional steps will be the same, and the result will be 10 once again.

Summing numbers with inject is a simple example of taking an array of numbers and building a resulting sum one element at a time.

Example 2: Building a Hash
Sometimes you'll have data in one format, but you really want it in another. For example, you may have an array that contains keys and values as pairs, but it's really just an array of arrays. In that case, inject is a nice solution for quickly converting your array of arrays into a hash.

hash = [[:first_name, 'Shane'], [:last_name, 'Harvie']].inject({}) do |result, element|
  result[element.first] = element.last
  result
end

hash # => {:first_name=>"Shane", :last_name=>"Harvie"}

As the example shows, I start with an empty hash (the argument to inject) and I iterate through each element in the array adding the key and value one at a time to the result. Also, since the result of the block is the next yielded result, I need to add to the hash, but explicitly return the result on the following line.

Ola Bini and rubikitch both pointed out that you can also create a hash from an array with the following code.

Hash[*[[:first_name, 'Shane'], [:last_name, 'Harvie']].flatten] # => {:first_name=>"Shane", :last_name=>"Harvie"}

Of course, I can do other things in inject also, such as converting the keys to be strings and changing the names to be lowercase.

hash = [[:first_name, 'Shane'], [:last_name, 'Harvie']].inject({}) do |result, element|
  result[element.first.to_s] = element.last.downcase
  result
end

hash # => {"first_name"=>"shane", "last_name"=>"harvie"}

This is a central value for inject, it allows me to easily convert an enumerable into an object that is useful for the problem I'm trying to solve.

Example 3: Building an Array
Enumerable gives you many methods you need for manipulating arrays. For example, if want all the integers of an array, that are even, as strings, you can do so chaining various methods from Enumerable.

[1, 2, 3, 4, 5, 6].select { |element| element % 2 == 0 }.collect { |element| element.to_s } # => ["2", "4", "6"]

Chaining methods of Enumerable is a solution that's very comfortable for many developers, but as the chain gets longer I prefer to use inject. The inject method allows me to handle everything I need without having to chain multiple independent methods.

The code below achieves the same thing in one method, and is just as readable, to me.

array = [1, 2, 3, 4, 5, 6].inject([]) do |result, element|
  result << element.to_s if element % 2 == 0
  result
end

array # => ["2", "4", "6"]

Of course, that example is a bit contrived; however, a realistic example is when you have an object with two different properties and you want to build an array of one, conditionally based on the other. A more concrete example is an array of test result objects that know if they've failed or succeeded and they have a failure message if they've failed. For reporting, you want all the failure messages.

You can get this with the built in methods of Enumerable.

TestResult = Struct.new(:status, :message)
results = [
  TestResult.new(:failed, "1 expected but was 2"),
  TestResult.new(:sucess),
  TestResult.new(:failed, "10 expected but was 20")
]

messages = results.select { |test_result| test_result.status == :failed }.collect { |test_result| test_result.message }
messages # => ["1 expected but was 2", "10 expected but was 20"]

But, it's not obvious what you are doing until you read the entire line. You could build the array the same way using inject and if you are comfortable with inject it reads slightly cleaner.

TestResult = Struct.new(:status, :message)
results = [
  TestResult.new(:failed, "1 expected but was 2"),
  TestResult.new(:sucess),
  TestResult.new(:failed, "10 expected but was 20")
]

messages = results.inject([]) do |messages, test_result|
  messages << test_result.message if test_result.status == :failed
  messages
end
messages # => ["1 expected but was 2", "10 expected but was 20"]

I prefer to build what I want using inject instead of chaining methods of Enumerable and effectively building multiple objects on the way to what I need.

Example 4: Building a Hash (again)
Building from the Test Result example you might want to group all results by their status. The inject method lets you easily do this by starting with an empty hash and defaulting each key value to an empty array, which is then appended to with each element that has the same status.

TestResult = Struct.new(:status, :message)
results = [
  TestResult.new(:failed, "1 expected but was 2"),
  TestResult.new(:sucess),
  TestResult.new(:failed, "10 expected but was 20")
]

grouped_results = results.inject({}) do |grouped, test_result|
  grouped[test_result.status] = [] if grouped[test_result.status].nil?
  grouped[test_result.status] << test_result
  grouped
end

grouped_results
# >> {:failed => [
# >>    #<struct TestResult status=:failed, message="1 expected but was 2">, 
# >>    #<struct TestResult status=:failed, message="10 expected but was 20">],
# >>  :sucess => [ #<struct TestResult status=:sucess, message=nil> ]
# >> }

You might be sensing a theme here.

Example 5: Building a unknown result
Usually you know what kind of result you are looking for when you use inject, but it's also possible to use inject to build an unknown object.

Consider the following Recorder class that saves any messages you send it.

class Recorder
  instance_methods.each do |meth|
    undef_method meth unless meth =~ /^(__|inspect|to_str)/
  end

  def method_missing(sym, *args)
    messages << [sym, args]
    self
  end

  def messages
    @messages ||= []
  end
end

Recorder.new.will.record.anything.you.want
# >> #<Recorder:0x28ed8 @messages=[[:will, []], [:record, []], [:anything, []], [:you, []], [:want, []]]>

By simply defining the play_for method and using inject you can replay each message on the argument and get back anything depending on how the argument responds to the recorded methods.

class Recorder
  def play_for(obj)
    messages.inject(obj) do |result, message|
      result.send message.first, *message.last
    end
  end
end

recorder = Recorder.new
recorder.methods.sort
recorder.play_for(String)

# >> ["<", "<=", "<=>", "==", "===", "=~", ">", ">=", "__id__", ...]

Conclusion
How do I know when I want to use inject? I like to use inject anytime I am building an object a piece at a time. In the case of summing, creating a hash, or an array I'm building a result by applying changes based on the elements of the enumerable. After I'm done applying changes for each element, I have the finished object I'm looking for. The same is true of the Recorder example, I'm sending the methods one at a time until I return the result of sending all the methods.

So next time you need to build an object based on elements of an enumerable, consider using inject.