Domain Specific Language Simplexity

There's one kind of simplicity that I like to call simplexity. When you take something incredibly complex and try to wrap it in something simpler, you often just shroud the complexity -- Anders Hejlsberg

At QCon London I caught the Domain Specific Language (DSL) tutorial by Martin Fowler, Neal Ford, and Rebecca Parsons. While Martin covered how and why you would want to create a DSL he discussed hiding complexity and designing solutions for specific problems. As Martin went into further detail all I could think was: Simplexity.

Anders prefers simple all the way down. Of course, hopefully we all prefer simplicity; however, the devil generally lives in the details.

Some things are complex by nature. Designing a ORM is a great example of a problem that I've yet to see a bullet proof solution for. Most ORMs are complex by necessity, but hiding that complexity in a DSL specific to your application is a good thing. I'll take simplexity over complexity pretty much any day.

I've designed several simplex Domain Specific Languages.

Domain Specific Flow Language
About a year and a half ago Tim Cochran and I designed a few objects that allowed you to define the flow of an application using code similar to the example below.

pages :customer, :shipping, :billing, :summary, :confirmation, :offer, :retailer

flow.customer_driven customer >> shipping >> billing >> summary >> confirmation
flow.retailer_driven retailer >> billing >> summary >> confirmation
flow.offer_driven offer >> shipping >> billing >> summary >> confirmation

The business led us to believe that the flow of the application was going to change often and it needed to be pliable. With that in mind Tim and I set out to create something that was simple to alter and didn't require complete understanding. We wrote out the syntax first and then made it execute. It was fairly easy for us to follow and we had a solution within a few hours. Then we presented the solution to the team.

We showed how to use the DSL and everyone loved it. I think it was actually most people's favorite part of the codebase. Then we dove into how it worked. I'm fairly sure that no one understood the underlying code. The solution was very simplex. It didn't really matter at the time because it was well tested and I don't remember it ever changing in the 6 months I was on the project.

About a year later Jake Scruggs wrote about how the current version of the Flow code is unnecessarily complex. I'm sure Jake is right. If the flows are rarely changing and the underlying code is causing any problems, it should definitely be removed.

The situation brings about an interesting question: how do you know when it's worth creating a DSL? In this example it was a requirement that it be easy to change a flow, but what is easy? I think the original implementation is easier to read and change, but much harder to fully understand and redesign. Unfortunately, there's no simple answer. Like so many things in software development, whether or not to introduce a DSL really depends on the context.

In this case, Tim and I decided to use a DSL because we expected the frequency of change to be high and there was a requirement that the time to make a change needed to be short. Given those requirements, something that is easy to read and change, but hides the underlying complexity is probably a good solution.

Expectations Delegation
I recently created a testing framework: expectations. One of the features of expectations is a solution for testing delegation. Delegation is such a commonly used pattern that I found it helpful to have an simple way to test that delegation is happening as expected.

The following code verifies that delegation does occur to the property that you expect and the result of the method you delegate to is returned from the delegating method.

expect PersonProxy.new.to.delegate(:name).to(:subject) do |proxy|
  proxy.name
end

In the example, the expectation ensures that the proxy.name method calls proxy.subject.name and the return value from proxy.subject.name is also returned by proxy.name.

The resulting expectation is very easy to read and write, but the underlying code is actually quite complex. First of all you need to stub the proxy.subject method. The return value of proxy.subject needs to be a mock that expects a name method call. If the mock receives the name method call you know that delegation happened correctly. The behavior based testing isn't overly complex, but it's not simple either, and it's the easy bit.

Ensuring that the result of proxy.subject.name is returned from proxy.subject is much more complicated. Part of the problem is that the call to proxy.name happens within the block. You could use the result of the block and compare it to what proxy.subject.name is supposed to return, but then the following code would fail.

expect PersonProxy.new.to.delegate(:name).to(:subject) do |proxy|
  proxy.name
  nil
end

You could argue that the above code should cause a failure, but if it does it creates some less than desirable results. First of all, the error message will need to stay something like "Delegation may have occurred, but the return value of the block returned something different so delegation cannot be verified". Also, comparing the result of the block for state based tests makes sense, but here we are only testing delegation so it's not intuitive that the result of the block is significant.

I solved this by specifying the return value of proxy.subject.name, dynamically defining a module that defines the name method, captures the result of the delegation and stores it for later verification, and then extending the proxy with the dynamically defined module. If that's a lot to take in, don't worry I've yet to meet anyone thought it was easy to follow.

Underlying complexity is not desirable. If I could find an easier way to test delegation, I would absolutely change my implementation. Unfortunately, no one has been able to come up with a better solution, yet.

I do think the current solution is better than having to write traditional repetitive and overly verbose code to test every delegation. The DSL for testing delegation allows you to focus on higher value problems. Whether you follow the underlying implementation or not, you must admit that being able to test delegation in a readable and reliable way is a good thing.

Sometimes, complexity is warranted when the resulting interface is simple enough that you gain productivity overall.

Conveying Intention and Removing Noise
Part of what introduces simplexity to a Domain Specific Language is the need to write what you want without introducing side effects. In the expectations example, the return block of the expectation shouldn't be used for comparison because it's not immediately obvious that it would be significant. Using the block return value for comparison is easier from an implementation perspective, but it makes the DSL less usable. This is not a good trade-off.

Noise is defined differently by everyone, but I like to think of it as the difference between what I would like to write and what I need to write.

Removing noise is also a common cause of simplexity. The following code is fairly easy to implement.

expect Process.new.to_have(:finished) do |process|
  process.finished = true
end

But, with expectations I chose to make the code below the legal syntax.

expect Process.new.to.have.finished do |process|
  process.finished = true
end

I chose the later version for a few reasons. The "to" method is the gateway method to all types of expectations. If you want a delegation expectation you write "expect Object.to.delegate...", if you want a behavior based expectation you write "expect Object.to.receive..", and if you want to use a boolean method defined on the class you can use "expect Object.to.be..." or "expect Object.to.have..." depending on what reads better. Using dots allows me to create a consistent interface for all the different expectations, and it also allows me to create expectations on any object without creating several different methods on the Object class.

I also chose to allow dot syntax because once you call "to.be" you can begin calling methods exactly as you would on the object itself. If the object is designed with fluency in mind the test can read as a dot delimited pseudo sentence. For example, the following test is an easy way to verify validation.

class Person < ActiveRecord::Base
  validates_presence_of :name
end

expect Person.new.to.have.errors.on(:name) do |person|
  person.valid?
end

To get this desired behavior I rely on the use of method_missing. Using the method_missing method almost always increases complexity by an order of magnitude. However, in this case, if I didn't use method missing I'd need a solution similar to the one below.

expect Person.new.to.be.true.when(:errors).then(:on, [:name]) do |person|
  person.valid?
end

While this version is easier to implement, it's much less friendly to use.

The level of simplexity should probably be defined by the usage of the Domain Specific Language. If the language is only used by a small subset of employees on few occasions then it might not make sense to increase simplexity. Also, if the complexity of the implementation is raised to a level that it cannot effectively be maintained, you absolutely must back off some. However, if the complexity is maintainable and the DSL is used frequently, I would do my best to reduce "noise".

Conclusion
I believe that simplexity is inevitable when designing a Domain Specific Language, and it shouldn't be something that stops you from using one. However, it's always valuable to weigh how much complexity has been introduced by the desire to create a simple user experience. If the complexity is more painful than the benefit to the consumers, you've clearly gone too far. However, if the complexity is painless, but the DSL is so unpleasant that it's usage is limited, you might need to work on designing more friendly DSL, even at the expense of simplicity.