Thursday, July 06, 2006

Ruby Continuous Integration

My new project is starting up and we needed to make a decision on what type of Continuous Integration we were going to use. On the previous project we used Bitten, but it constantly caused us problems. We were going to use DamageControl, but it doesn't quite appear to be ready for use. We even tried using the Continuous Builder plugin, but it causes Subversion to hang while it runs the build.

So, what's left? Rolling your own, of course. I'm not a big fan of re-inventing the wheel, but it seemed like my other options were still prototype or busted wheels.

We chose a fairly simple solution. We are using Subversion, so we added the following post-commit hook:
#!/bin/sh
sudo -H -u builder /bin/bash --login -c "cd /builds/commits && /usr/bin/touch $2"
This creates a file from the revision number passed as an argument to post-commit.

Then, in the /builds folder we have builder.rb
require File.expand_path(File.dirname(__FILE__) + "/builder_config")
require 'fileutils'
require 'rubygems'
require 'ruby-growl'

loop do
unbuilt = Dir["#{BuilderConfig.commits_folder}/*"].collect { |path| File.basename(path).to_i }.sort
exit if unbuilt.size == 0
latest_revision_number = unbuilt.pop
FileUtils.rm "#{BuilderConfig.commits_folder}/#{latest_revision_number}"
FileUtils.cd "#{BuilderConfig.source_folder}"
system "svn update -r#{latest_revision_number}"
FileUtils.mkdir_p "#{BuilderConfig.output_folder}/#{latest_revision_number}"
status = system("rake > #{BuilderConfig.output_folder}/#{latest_revision_number}/rake-output.log") ? :success : :failure
system "svn log -r#{latest_revision_number} -v > #{BuilderConfig.output_folder}/#{latest_revision_number}/svn-log.log"
system "touch #{BuilderConfig.output_folder}/#{latest_revision_number}/#{status.to_s}"
['10.0.1.1','10.0.1.2'].each do |ip|
devbox = Growl.new ip, 'Builder', ['success', 'failure'], nil, 'grrbabygrr'
devbox.notify status.to_s, "Build #{latest_revision_number} #{status.to_s}", ""
end
end
Builder.rb loops through the commits folder pulling all files that are in commits. When it finds a file it removes it and stores the latest_revision_number. It then updates the code to that revision, creates a directory for that revision in the output, runs rake and saves the output to the revision folder, saves the svn info to the revision output folder, saves a file called success or failure, and sends a Growl message to the pairing stations. We use cron to run builder.rb every minute. It's very low tech, but it seems to work well for us.

For displaying build results we created a rails project that's quite simple. All we needed was one controller to pull the file info from the /builds dir, one view to display the info, and one model to logically group data from each build.

Like I said, I'm not a big fan of rolling your own solutions, but it's nice when you are using tools that let you easily come up with a solution.

16 comments:

  1. I feel your pain. I ended up rolling my own several weeks ago for exactly the same reasons. Guess I should have posted it somewhere and saved you the trouble. ;-) Just been too busy.

    If you're interested in taking a look at what I used, you can get the source at svn://poseidon.johnwilger.com/nsa/trunk

    ReplyDelete
  2. If you background the CI plugin w/ an ampersand it'll let your commit finish and run in the background. You then have to setup sudo and a couple other things that pretty much make it a pita no matter what. Nice CI though and bonus points for using the growl notifier. :)

    ReplyDelete
  3. atmos:

    That was not my experience.

    No manner of ampersands, nohup'ing or the like would force the process into the background.

    A bit of digging around turned up that certain releases of Subversion would not allow the post-commit to exit if any child process still had an open connection to the STDOUT or STDERR that was opened by the post-commit hook.

    If the ampersand works for you, it is because you are using a different version of Subversion on your repo server.

    ReplyDelete
  4. Anonymous12:16 PM

    Why not just use CruiseControl or CC.Net just to monitor SVN and kick off the rake script?

    ReplyDelete
  5. CC and CC.net are great, but a lot more than we needed. It was easier to roll our own than it was to set up either CC implementations to work with ruby.

    That said, I'd love to see first class support for Ruby in both implementations.

    ReplyDelete
  6. Hi, Jay.

    I suggest you to try Cerberus CI tool. It is easy-to-use tool that do what you need.

    http://rubyforge.org/projects/cerberus

    Main advantages over other solutions:
    = Cerberus could be installed on any machine not only where SVN repository located.
    = Cerberus works not only for Rails projects, but for any other Ruby (or better to say for projects that use Rake)
    = Cerberus multyplatform solution: it runs excellent both on *nix and Windows.
    = Cerberus distributed via RubyGems, so it is very easy to install and very easy to update to latest versin
    = Cerberus very easy to use 'cerberus add PROJECT_URL|PROJECT_DIR'
    = Cerberus is lightweigt solution: mots of the time ruby process even not run - Rake runs only in case if changes in project found
    and then you need to run from Cron
    'cerberus build PROJECT_NAME'

    Just try to instal (by 'sudo gem install cerberus') and use and you'll see how is CI for ruby easy.

    ReplyDelete
  7. Thanks for the tip. What we have works now so I don't think we'll take the effort to change; however, hopefully someone else can take your suggestion.

    ReplyDelete
  8. If your solution works for you then probably there is no sense to change.

    But for new projects I highly recommend you Cerberus. It was created with 'easy-in-use' idea in mind. So it is very easy to pick up and start using it.

    ReplyDelete
  9. If you happened to have two commits in one minute, this code looks like it will update and build the latest revision first, then loop again and update and build the older revision. However, for a working copy on r1, "svn update -r3" would also suck down the sources updated in r2, so the update and build for r2 would essentially be the same as for r3.

    Am I reading this right? If so, that's cool, I'll just assume you won't likely have two commits in one minute and therefore it doesn't matter.

    ReplyDelete
  10. Hello Shawn,

    The build for r3 would include the changes for r2; however, the build for r2 would not include r3's changes. Therefore if two people commited at the same time and r3 broke the build, but then r2 broke the build also you should be able to easily identify it was the changes in r2 that likely are causing an issue.

    ReplyDelete
  11. Shawn8:20 AM

    Indeed, I stopped one step shy of realizing that updating to r2 will roll back r3's changes. Thanks for the clarification.

    And thanks for an extremely practical blog. The code examples in the unit testing/stub/mock posts and others like this one are fantastic learning tools!

    ReplyDelete
  12. Jay,

    ThoughtWorks has a nice list of continuous integration servers ant it is not limited to DamageControl: http://damagecontrol.codehaus.org/Continuous+Integration+Server+Feature+Matrix

    You might want to check it before rolling your own.

    Hope this helps.

    Regards,

    Slava Imeshev

    ReplyDelete
  13. Slava: We rolled our own solution in about 1 week with four guys. It was quite simple.

    As far as packaged solutions, I've not heard of a single one that works well for Ruby. We recently used Cerberus and it's very immature and easier to roll your own. I've heard the same comments about other options. I appreciate the link, but until I hear some success stories with any given solution, I'm skeptical.

    Thanks for the comment.

    ReplyDelete
  14. Just to let you know, I have drawn a statechart diagram in an attempt to represent the Continuous Integration workflow

    http://21croissants.blogspot.com/2006/12/continuous-integration-workflow.html

    I thought you experts could comment on it,

    Jay, thanks for your blog!

    Jean-Michel

    ReplyDelete
  15. A lot has transpired since this post, but just for those coming here from search engines... one option for continuous integration would be hudson, easy to setup... steps here:
    http://blog.huikau.com/2008/01/09/jruby-ruby-continuous-integration-with-hudson/

    ReplyDelete
  16. Hi! I just published a very simple Continuous Integration App for RubyOnRails + SVN/GIT. Maybe you should give it a try:

    http://github.com/felipegiotto/Inotegration/tree/master

    I took some ideas of measurement and testing tools and, after trying some CI tools and not liking anyone of them, I decided to build my own, without needing to build big XML files or any other configuration. Just the way Rails was made to be.

    If you like, please send me some feedback..

    PS: Sorry for using your blog to send this, but I just want to make a good CI tool for the Rails Community.

    Best regards,

    Felipe Giotto.

    ReplyDelete

Note: Only a member of this blog may post a comment.