Business Natural Languages:
IntroductionBusiness Natural Languages:
The ProblemBusiness Natural Languages:
The SolutionBusiness Natural Languages:
DRY code, DAMP BNLBusiness Natural Languages: Moving to the web and multiple contexts
Business Natural Languages:
LimitationsBusiness Natural Languages:
Common PiecesIn 'The Solution' we proved our concept by replacing the existing system with one that allows the business logic to be written by the subject matter experts. However, the solution we provided has a few limitations our client would like us to overcome:
- Our application can only be run from the command line, they would like a web interface.
- Our application only executes compensation calculations for the entire group; they would like to execute individual compensation calculations.
- The sales data that was previously stored in yaml files should now be read from their database.
- Our application does not scale well; they would like the burden of calculation to be transferred to their database server when running for every employee.
Now that we are required to connect to the database and move to the web, Ruby on Rails is the obvious solution. This book assumes familiarity with creating a new Ruby on Rails applications. If you need more info on Ruby on Rails applications I suggest 'Agile Web Development with Rails' by Dave Thomas and DHH.
For those coding along, the next steps are:
1. create a Rails application named payroll
2. Set up a dev database and alter database.yml to point to your new database.
3. Add the following code to environment.rb:
Inflector.inflections do |inflect|
inflect.uncountable %w( employee_info )
end
We also need to create the sales data that we would expect to be available when our application executes. Creating the sales tables and data is accomplished with the CreateSalesData Migration class.
class EmployeeInfo < ActiveRecord::Base; end
class CreateSalesData < ActiveRecord::Migration
def self.up
create_table :employee_info, :force=>true do |table|
table.column :employee, :string
table.column :deals_this_month, :integer
table.column :year_old_deals, :integer
table.column :gross_profit, :integer
end
EmployeeInfo.create(:employee=>'jjones',
:deals_this_month=>7,
:year_old_deals=>2,
:gross_profit=>1400000)
EmployeeInfo.create(:employee=>'jjohnson',
:deals_this_month=>12,
:year_old_deals=>1,
:gross_profit=>2200000)
end
def self.down
drop_table :employee_info
end
end
After creating the sales data you can create the following SalesInfo class that represents the facade to the sales data. Note: The SalesInfo class still returns EmployeeInfo instances, and EmployeeInfo has now become an ActiveRecord::Base derived class.
class SalesInfo
def self.method_missing(sym, *args)
result = EmployeeInfo.find_by_employee sym.to_s
super if result.nil?
result
end
end
class EmployeeInfo < ActiveRecord::Base
end
With the sample data taken care of, the first thing necessary to accommodate the new requirements is create a table to store the compensation logic.
class CreateCompensationScripts < ActiveRecord::Migration
def self.up
create_table :compensation_scripts do |table|
table.column :logic, :string, :limit=>1000
end
end
def self.down
drop_table :compensation_scripts
end
end
We are also going to need a few views, a model, and a controller to allow the business users to input new compensation logic.
File: new.rhtml
<% form_for :script, @script, :url => { :action => "create" } do |form| %>
Compensation Script<br>
<%= form.text_area :logic %><br>
<%= submit_tag %>
<% end %>
File: index.rhtml
Employees:
<ul>
<% @scripts.each do |script| %>
<li>
<%= script.employee_name %> -
<%= link_to 'view', :action => :view, :id => script.id %> |
<%= link_to 'execute', :action => :execute, :id => script.id %>
</li>
<% end %>
</ul>
<%= link_to 'Execute All', :action=>:execute_all %> |
<%= link_to 'Create new compensation script', :action=>:new %>
File: view.rhtml
<pre><%= @script.logic %><pre>
<%= link_to 'back', :action => :index %>
File: compensation_script.rb
class CompensationScript < ActiveRecord::Base
def employee_name
logic.split(/\n/).first.gsub(/^employee\s/, '').chomp
end
def employee_short_name
name_parts = employee_name.split
(name_parts.first[0,1] + name_parts.last).downcase
end
end
File: payroll_controller.rb
class PayrollController < ApplicationController
def index
@scripts = CompensationScript.find :all
end
def new
@script = CompensationScript.new
end
def create
CompensationScript.create(params[:script])
redirect_to :action => :index
end
def view
@script = CompensationScript.find(params[:id])
end
def execute
script = CompensationScript.find(params[:id])
vocabulary = InlineVocabulary.new(script.employee_short_name)
@compensation = CompensationParser.parse(script.logic, vocabulary, InlineContext.new)
end
def execute_all
@compensations = []
CompensationScript.find(:all).each do |script|
vocabulary = SqlVocabulary.new(script.employee_short_name)
@compensations << CompensationParser.parse(script.logic, vocabulary, SqlContext.new)
end
end
end
As you can see there is logic for executing also included; however, for the time being let's focus on adding our compensation logic to the application. To store the compensation logic navigate to http://localhost:3000/payroll and click the 'Create new compensation script' link, then enter the following logic for John Johnson.
employee John Jones
compensate $2500 for each deal closed in the past 30 days
compensate $500 for each active deal that closed more than 365 days ago
compensate 5% of gross profits if gross profits are greater than $1,000,000
compensate 3% of gross profits if gross profits are greater than $2,000,000
compensate 1% of gross profits if gross profits are greater than $3,000,000
Repeat the previous steps and enter in the logic for Jackie Johnson.
employee Jackie Johnson
compensate $3000 for each deal closed in the past 30 days
compensate $800 for each active deal that closed more than 365 days ago
compensate 5% of gross profits if gross profits are greater than $1,000,000
compensate 5% of gross profits if gross profits are greater than $2,000,000
After the logic is saved to the database you should be able to click the 'view' link from the index page to verify that the logic has been entered correctly.
Now, back to the previously mentioned execute logic. The index page has an 'execute' link that links to a page that doesn't yet exist. The execute.rhtml file is a simple page that shows the employee's name and their calculated bonus:
File: execute.rhtml
<%= @compensation.name %> compensation: <%= number_to_currency @compensation.amount %><br>
<%= link_to 'back', :action => :index %>
A brief look in the PayrollController class reveals that execute method relies on the InlineVocabulary, CompensationParser, and InlineContext classes to create a parse tree from the logic stored in a CompensationScript. The InlineVocabulary has been previously shown as the CompensationVocabulary class. It has been renamed to clearly express intent and allow the SqlVocabulary class to be defined.
File: inline_vocabulary.rb
class InlineVocabulary
extend Vocabulary
def initialize(data_for)
@data_for = data_for
end
def logic_initializer
''
end
phrase "active deal that closed more than 365 days ago!" do
SalesInfo.send(@data_for).year_old_deals.to_s
end
phrase "are greater than" do
" > "
end
phrase "deal closed in the past 30 days!" do
SalesInfo.send(@data_for).deals_this_month.to_s
end
phrase "for each" do
"*"
end
phrase "gross profits" do
SalesInfo.send(@data_for).gross_profit.to_s
end
phrase "if" do
" if "
end
phrase "of" do
"*"
end
end
The Vocabulary module, which InlineVocabulary extends, remains unchanged; however, I will show it again for completeness.
File: vocabulary.rb
module Vocabulary
def phrase(name, &block)
define_method :"_#{name.to_s.gsub(" ","_")}", block
end
end
The InlineContext class introduces another layer of abstraction. This is necessary to allow the compensation to be evaluated in various contexts. When the execute method is invoked the InlineVocabulary will create valid Ruby code, thus allowing the InlineContext class to simply call instance_eval to evaluate and return.
File: inline_context.rb
class InlineContext
def evaluate(logic)
instance_eval(logic)
end
end
Processing the Business Natural Language is basically the same as the previous command line application; therefore, the CompensationParser class remains largely the same as the previously shown example. The only change to the class is that the parse method now accepts a context and passes this context to the instance of the Root object that is created.
File: compensation_parser.rb
class CompensationParser
class << self
def parse(script, vocabulary, context)
root = Root.new(vocabulary, context)
script.split(/\n/).each { |line| root.process(preprocess(line)) }
root
end
def preprocess(line)
line.chomp!
line.delete!('$,')
line.gsub!(/(\d+)%/, '\1percent')
line.gsub!(/\s/, '._')
"_#{line.downcase}!"
end
end
end
Similar to the CompensationParser class, the Root class also remains largely the same notwithstanding taking a context instance as a constructor argument and passing it to each newly created instance of the Compensation class.
File: root.rb
class Root
extend Vocabulary
def initialize(vocabulary, eval_context)
@compensations = []
@vocabulary, @eval_context = vocabulary, eval_context
end
def name
@employee.name
end
def amount
@compensations.collect do |compensation|
compensation.amount
end.inject { |x, y| x + y }
end
def process(line)
instance_eval(line)
end
phrase :employee do
@employee = Employee.new
end
phrase :compensate do
@compensations << Compensation.new(@vocabulary, @eval_context)
@compensations.last
end
end
The Root class also uses the unchanged Employee class.
File: employee.rb
class Employee
def initialize
@name_parts = []
end
def method_missing(sym,*args)
@name_parts << sym.to_s.delete('_!')
self
end
def name
@name_parts.collect { |part| part.to_s.capitalize }.join(' ')
end
end
The Compensation class also basically mirrors it's previously show version. A minimal change now initializes the compensation_logic instance variable to the value returned from the logic_initializer method of the vocabulary construtor argument. The logic_initializer method of the InlineVocabulary is an empty string; however, we will see later how the SqlVocabulary class uses this to create valid SQL statements. Another notable change to the Compensation class allows for the logic to be executed in various ways. This subtle change can be found in the amount method of the Compensation class. By delegating the evaluation to another class we can simply instance_eval, which InlineContext does, or pass the burden of evaluation on to another process, which will be seen within the SqlContext class.
Take a minute to contemplate this and the previous paragraph if you don't yet appreciate the value of various contexts. Various contexts allow you to evaluate the same code and produce different results based on your current needs. This can be used to delegate responsibility to an external application (e.g. Database server), external process (e.g. a process written in another language), or simply evaluate the code within your application in another way (e.g. Syntax checking). When using Domain Specific Languages, context is king or at least royalty.
Getting back on track, if you've been typing along you should now be able to execute the logic you entered for both John Jones and Jackie Johnson. At this point we have fulfilled the first three of the new requirements. A quick check reveals that both totals report the same numbers previously generated.
John Jones compensation: $88,500.00
Jackie Johnson compensation: $256,800.00
To accommodate the last requirement we are going to create a new page that displays the return values from the database, use the previously shown execute_all method of the PayrollContorller class, create the SqlVocabulary class, and create the SqlContext class which will be used to evaluate the SQL statements generated by the SqlVocabulary class.
The execute_all.rhtml file is responsible for displaying the results of executing the compensation logic for all employees.
File: execute_all.rhtml
<% @compensations.each do |compensation| %>
<%= compensation.name %> compensation: <%= number_to_currency compensation.amount %><br>
<% end %>
<%= link_to 'back', :action => :index %>
The execute_all method of the PayrollController is similar to the execute method except it iterates through each CompensationScript and appends the results in an instance variable array. Another difference is the use of the SqlVocabulary and SqlContext classes. The SqlVocabulary class behaves exactly the same as the InlineVocabulary class, but returns fragments of SQL statements instead of Ruby code. The SqlVocabulary class also defines the logic_initializer method to return 'select ', the necessary prefix for the SQL statements we will generate.
File: sql_vocabulary.rb
class SqlVocabulary
extend Vocabulary
def initialize(data_for)
@data_for = data_for
end
def logic_initializer
'select '
end
def select_column(column)
" (select #{column} from employee_info where employee = '#{@data_for}') "
end
phrase "active deal that closed more than 365 days ago!" do
select_column("year_old_deals")
end
phrase "are greater than" do
" > "
end
phrase "deal closed in the past 30 days!" do
select_column("deals_this_month")
end
phrase "for each" do
"*"
end
phrase "gross profits" do
select_column("gross_profit")
end
phrase "if" do
" where "
end
phrase "of" do
"*"
end
end
The implementation of the SqlContext class uses the execute method of ActiveRecord::Base.connection. This anticlimactic implementation allows us to easily delegate evaluation to the database. I've used this pattern very successfully on two different projects that required calculation and modification of hundreds of millions of records within a few hours.
File: sql_context.rb
class SqlContext
def evaluate(logic)
result = ActiveRecord::Base.connection.select_value logic
result.to_i unless result.nil?
end
end
If you are still typing along you should now be able to navigate from the index page using the 'Execute All' link to a page that shows the following results.
John Jones compensation: $88,500.00
Jackie Johnson compensation: $256,800.00
This doesn't appear very impressive since we already knew how to calculate the results. However, the previously mentioned point of interest is that instead of calculating the results using ruby, the results are calculated from generated sql statements. For example, below is the SQL generated to calculate Jackie Johnson's bonus.
select 3000* (select deals_this_month from employee_info where employee = 'jjohnson')
select 800* (select year_old_deals from employee_info where employee = 'jjohnson')
select 0.05* (select gross_profit from employee_info where employee = 'jjohnson') where (select gross_profit from employee_info where employee = 'jjohnson') > 1000000
select 0.05* (select gross_profit from employee_info where employee = 'jjohnson') where (select gross_profit from employee_info where employee = 'jjohnson') > 2000000
An important point to note is the that when Jackie Johnson's bonus terms change both the results for execute and execute_all will be updated by the single change that the subject matter expert will make. Obviously, system maintainability has been increased.
At this point we've satisfied the final requirement for applicaiton. The next chapter will focus on some of the limitations of our current implementation and Business Natural Languages in general.
