Jay Fields' Thoughts

Following my blog entry Underuse of Modules, an anonymous commenter asks

Can you show another example, then, of how one might implement the "magic" of Dwemthy's Array (http://poignantguide.net/dwemthy/) just using modules? I can never remember how to do this sort of thing, and if modules can make it conceptually simpler it would be most useful.

I'm not sure exactly what magic they were referring to, but I'll assume they mean what allows creature habits to be defined in class definitions. Based on that assumption, I pulled out this code from the example.

class Creature
  def self.metaclass; class << self; self; end; end

  def self.traits( *arr )
    return @traits if arr.empty?

    attr_accessor(*arr)

    arr.each do |a|
      metaclass.instance_eval do
        define_method( a ) do |val|
          @traits ||= {}
          @traits[a] = val
        end
      end
    end

    class_eval do
      define_method( :initialize ) do
        self.class.traits.each do |k,v|
          instance_variable_set("@#{k}", v)
        end
      end
    end
  end
end

class Rabbit < Creature
  traits :bombs
  bombs 3
end

Rabbit.new.bombs # => 3

Note, I removed the comments and added the Rabbit code. The Rabbit is there to ensure the magic continues to function as expected.

The version using a module isn't significantly better, but I do slightly prefer it.

class Creature
  def self.traits( *arr )
    return @traits if arr.empty?

    attr_accessor(*arr)

    mod = Module.new do
      arr.each do |a|
        define_method( a ) do |val|
          @traits ||= {}
          @traits[a] = val
        end
      end
    end

    extend mod
    
    define_method( :initialize ) do
      self.class.traits.each do |k,v|
        instance_variable_set("@#{k}", v)
      end
    end
  end
end

class Rabbit < Creature
  traits :bombs
  bombs 3
end

Rabbit.new.bombs # => 3

The above version is a bit clearer to me because it defines methods on a module and then extends the module. I know that if I extend a module from the context of a class definition the methods of that module will become class methods.

Conversely, the first example forces me to think about where a method goes if I do an instance_eval on a metaclass. By definition all class methods are instance methods of the metaclass, but there are times when you can be surprised. For example, using def changes based on whether you use instance_eval or class_eval, but define_method behaves the same (change metaclass.instance_eval in the original example to metaclass.class_eval and the behavior doesn't change). This type of thing is an easy concept that becomes a hard to find bug.

If you spend enough time with metaclasses it's all clear and easy enough to follow. However, modules are generally straightforward and get you the same behavior without the mental overhead. I'm sure someone will argue that metaclasses are easier to understand, which is fine. Use what works best for you.

However, there are other reasons why it might make sense to use modules instead, such as wanting to have an ancestor (and thus the ability to redefine and use super).

Again, it probably comes down to personal preference.

Labels: extend, include, metaclass, metaprogramming, module

When I began seriously using Ruby I noticed two things that I didn't like about the language.

• There's no method to get the metaclass.
• Some people create modules as a hint that they reopened the class.

It's a few years later and I've noticed a few interesting things.

• I can't remember the last time I actually wanted a method to get the metaclass.
• If you use a module to add behavior your behavior becomes part of the ancestor tree, which is significantly more helpful than putting your behavior directly on the class.

No metaclass method necessary
In April 2005, Why gave us Seeing Metaclasses Clearly. I'm not sure the article actually helped me see metaclasses clearly, but I know I pasted that first block of code into several of my first Ruby projects. I used metaclasses in every way possible, several of which were probably inappropriate, and saw exactly what was possible, desirable, and painful.

I thought I had a good understanding of the proper uses for metaclasses, and then Ali Aghareza brought me a fresh point of view: defining methods on a metaclass is just mean.

We were on the phone talking about who knows what, and he brought up a blog entry I'd written where I dynamically defined delegation methods on the metaclass based on a constructor argument. He pointed out that doing so limited your ability to change the method behavior in the future. I created some simple examples and found out he was right, which lead to my blog entry on why you should Extend modules instead of defining methods on a metaclass.

Ever since that conversation and the subsequent blog entry, I've been using modules instead of accessing the metaclass directly. "Just in case someone wants to redefine behavior" isn't really a good enough reason for me if the level of effort increases, but in this case I found the code to be easier to follow when I used modules. In programming, there are few win-win situations, but Ali definitely showed me one on this occasion.

If you interact with a metaclass directly, do a quick spike where you introduce a module instead. I think you'll be happy with the resulting code.

Include modules instead of reopening classes
In January of 2007 I wrote a blog entry titled Class Reopening Hints. I didn't write it because I thought it was very valuable, I wrote it so developers afraid of open classes could get some sleep at night. Those guys think we are going to bring the end of the world with our crazy open classes, and I wanted to let them know we'd at least thought about the situation.

I'm really not kidding, I thought the entry was a puff piece, but it made some people feel better about Ruby so I put it together. I never followed the Use modules instead of adding behavior directly advice though, and I don't think many other Rubyists did either. It was extra effort, and I didn't see the benefit. In over two and an half years working with Ruby I've never once had a problem finding where behavior was defined. With that kind of experience, I couldn't justify the extra effort of defining a module -- until the day I wanted to change the behavior of Object#expects (defined by Mocha). I was able to work around the fact that Mocha defines the expects method directly on Object, but the solution was anything but pretty.

It turns out, using modules instead of adding behavior directly to a reopened class has one large benefit: I can easily define new behavior on a class by including a new module. If you only need new behavior, then defining a new method on the class would be fine. But, if you want to preserve the original behavior, having it as an ancestor is much better.

Take the following example. This example assumes that a method hello has been defined on object. Your task is to change the hello method to include the original behavior and add a name.

# original hello definition
class Object
  def hello
    "hello"
  end
end

# your version with additional behavior
class Object
  alias old_hello hello
  def hello(name)
    "#{old_hello} #{name}"
  end
end

hello("Ali") # => "hello Ali"

That code isn't terrible. In fact, there are a few different ways to redefine methods and access the original behavior, but none of them look as nice as the following example.

# original hello definition
class Object
  module Hello
    def hello
      "hello"
    end
  end
  include Hello
end

# your version with additional behavior
class Object
  module HelloName
    def hello(name)
      "#{super()} #{name}"
    end
  end
  include HelloName
end

hello("Ali") # => "hello Ali"

When you have an ancestor, the behavior is only a super call away.

note:Yes, I've also reopened the class to include the module, but when I talk about reopening the class I'm talking about defining the behavior directly on the reopened class. I could have also included the module by using Object.send :include, HelloName. Do whichever you like, it's not pertinent to this discussion.

Prefer Modules to metaclasses and reopened classes
The previous example illustrates why you should prefer modules, it gives simple access to your method behavior to anyone who wishes to alter but reuse the original behavior. This fact applies to both classes that include modules and instances that extend modules.

So why didn't Matz give us first class access to the metaclass? Who cares. He probably knew extending modules was a better solution, but even if he didn't -- it is. He didn't give you a method to access the metaclass, and whether he knew it or not, you don't need it.

Labels: extend, include, metaclass, module

I was recently working on the integration between Mocha and expectations. Expectations promotes the idea that you should only have one expectation per test; therefore, I wanted to display a warning if you call Object#expects within an expected block.

For example, the following code will print a warning.

Expectations do
  expect 1 do
    Object.expects(:something).returns 1
    Object.something
  end
end
# >> Expectations allows you to to create multiple mock expectations, but suggests that you write another test instead.
# >> expects method called from /Users/jay/example.rb:6
# >>
# >> Expectations .
# >> Finished in 0.001 seconds
# >> 
# >> Success: 1 fulfilled

Usually, I'd use one of the various Alternatives for Redefining Methods, but redefining the Object#expects method had two additional constraints that complicated matters.

• There's no (reasonable) way to extend all instances with a new module since the expects method is defined on Object.
• I didn't want to unconditionally redefine the expects method. Within the framework I call Object#expects, and I don't want those calls to cause invalid warnings. I need a solution that prints a warning when you use Object#expects within an expectation block, but does not print a warning if Object#expects is called from anywhere else.

At first I attempted to solve this problem by aliasing methods and pointing to different method definitions based on the context in which the code was currently executing. This turned into a complicated mess, and also required me to define a few methods that could potentially collide with methods defined by applications using Expectations. After going down that path for a bit it became clear that it would be simpler to define modules and delegate the expects call to the appropriate module based on the context.

The actual implementation involves several moving parts, so here's a much simpler example. Start with some behavior defined on Object. This behavior will have been defined by another framework so you cannot (easily) alter the original method definition.

class Object
  def say_hello
    "hello"
  end
end

Next, you've decided to create a framework that says hello in Spanish also. You still want to be able to return "hello" when English is required, but you want the say_hello method to return "hola" when you are expecting Spanish.

Below is the output we are looking for.

in_english do
  say_hello # => "hello"
end

in_spanish do
  say_hello # => "hola"
end

Currently our code returns "hello" both in English and Spanish.

# Framework Object::say_hello
class Object
  def say_hello
    "hello"
  end
end

# Your Object class
class Object
  def in_english(&block)
    instance_eval(&block)
  end
  
  def in_spanish(&block)
    instance_eval(&block)
  end
end

in_english do
  say_hello # => "hello"
end

in_spanish do
  say_hello # => "hello"
end

We can make the say_hello message sent to Object return "hola" by removing the say_hello method, defining an InSpanish module, and including the InSpanish module.

# Framework Object::say_hello
class Object
  def say_hello
    "hello"
  end
end

# Your Object class
class Object
  module InSpanish
    def say_hello
      "hola"
    end
  end
  include InSpanish
  
  remove_method :say_hello
  
  def in_english(&block)
    instance_eval(&block)
  end
  
  def in_spanish(&block)
    instance_eval(&block)
  end
end

in_english do
  say_hello # => "hola"
end

in_spanish do
  say_hello # => "hola"
end

Now we have Spanish working, but we've lost our English. Remember the actual implementation needed to preserve the original behavior in some circumstances. The in_english method is our circumstance where we need to preserve original behavior. This can be done easily enough by Moving the say_hello definition from Object to an InEnglish module.

# Framework Object::say_hello
class Object
  def say_hello
    "hello"
  end
end

# Your Object class
class Object
  module InSpanish
    def say_hello
      "hola"
    end
  end
  include InSpanish
  
  module InEnglish
    expects_method = Object.instance_method(:say_hello)
    define_method :say_hello do |*args|
      expects_method.bind(self).call(*args)
    end
  end
  include InEnglish

  remove_method :say_hello
  
  def in_english(&block)
    instance_eval(&block)
  end
  
  def in_spanish(&block)
    instance_eval(&block)
  end
end

in_english do
  say_hello # => "hello"
end

in_spanish do
  say_hello # => "hello"
end

Now we have the original behavior of the say_hello method, but we've lost our ability to speak Spanish.

The final step is to define Object#say_hello in a way that delegates the say_hello message to the appropriate module instead of removing the method.

# Framework Object::say_hello
class Object
  def say_hello
    "hello"
  end
end

# Your Object class
class Object
  module InSpanish
    def say_hello
      "hola"
    end
  end
  include InSpanish
  
  module InEnglish
    expects_method = Object.instance_method(:say_hello)
    define_method :say_hello do |*args|
      expects_method.bind(self).call(*args)
    end
  end
  include InEnglish

  def say_hello
    (@language || InEnglish).instance_method(:say_hello).bind(self).call
  end
  
  def in_english(&block)
    @language = InEnglish
    instance_eval(&block)
  end
  
  def in_spanish(&block)
    @language = InSpanish
    instance_eval(&block)
  end
end

in_english do
  say_hello # => "hello"
end

in_spanish do
  say_hello # => "hola"
end

The final change was setting the @language instance variable to the module who's say_hello method definition was required. As you can see from the printed output, our code works as desired.

This isn't a technique that you'll use often, but it's a good trick to know when you need it. If you're interested in the actual application you can check out the expectations framework code.

Labels: define_method, module

I was recently working with a framework that reopened and defined a method on Object. I wanted the behavior of the framework, but I also wanted to define my own behavior. This is generally the case for alias_method_chain or one of the other Alternatives for Redefining Methods, but my circumstances (to be discussed in a subsequent post) prevented me from using one of the better known solutions.

The solution that worked for me was to move the original method definition to a module.

class Speaker
  def say_hello
    "hello"
  end
end

module EnglishSpeaker
  expects_method = Speaker.instance_method(:say_hello)
  define_method :say_hello do |*args|
    expects_method.bind(self).call(*args)
  end
end

class Speaker
  undef say_hello
end

Speaker.new.say_hello # => -:18: undefined method `say_hello' for #<Speaker:0x285b4> (NoMethodError)
Speaker.new.extend(EnglishSpeaker).say_hello # => "hello"

You may never need this technique, I only needed it once in the past 2.5 years. But, when it applied I found it to be significantly better than the alternatives.

Labels: define_method, module, ruby

In my previous entry about Class Definitions Aman King left the following comment:

I've always liked Ruby's open class feature but have also wondered how do you keep track of all the changes made to the class definition during runtime? ...

There isn't an easy way (that I know of) out of the box to know what files made what changes to your objects. But, are a few things that you can do that may give you hints as to where behavior is coming from.

Use modules instead of adding behavior directly.
On my current project we needed to add a % method to symbol. You can accomplish this with the following code.

class Symbol
  def %(arg)
    ...
  end
end

The above code does what you need, but leaves no hint that you've made a change to Symbol. An alternative solution is to define a module and include that module in Symbol.

module SymbolExtension
  def %(arg)
    ...
  end
end

Symbol.send :include, SymbolExtension

Granted, this isn't a huge hint, but if you check Symbol.ancestors you'll find the following list.

Symbol
SymbolExtension
Object
Kernel

Many people believe this is better than nothing, but the choice is yours.

Use the stack trace.
Another option is to use the stack trace to try to track down a specific method. If you are working with some code where a method is being modified somewhere, but it isn't in a module the following code could help you out.

class Symbol
  def self.method_added(method)
    raise caller.to_s if method == :%
  end
end

class Symbol
  def %(arg)
    self
  end
end

The above code uses the method_added hook method and raises and exception when the method you are looking for is defined. At this point the stack trace should show you where the method was defined.

Use conventions.
In practice I've had very little trouble with finding where behavior is defined. In all the projects I've been involved with we have a few conventions that we follow that help ensure we can find behavior definitions easily.

• Define behavior for the class PhoneNumber (for example) in the phone_number.rb file
• If you need to open a class that you have not created, create a [Class]Extension module and include it as shown above.
• Avoid dynamically adding behavior in unexpected places when possible. For example we don't define behavior in associations, such as:

  # not recommended
  has_many :people do
    def by_name
      find(:all)
    end
  end

As I previously stated, I'm on a 14 person team currently. Those 14 people create quite a few features at a rapid pace. Despite our size and rapid pace, I can't remember ever having trouble finding behavior.

Labels: caller, class, extension, include, module, open, ruby, stack trace