Monday, June 23, 2008

Flex: The State of Testing

Given my focus on testing, it's not surprising that the most common question I get about Flex is: Is it testable? Of course, the answer is yes otherwise I wouldn't be using it.

Out of the box, it's very similar to early JUnit. Like all XUnit ports, it's based on creating testing classes and defining methods that are tests. There's setup, teardown, and all the usual features you expect from XUnit ports.

There's a FlexUnit swc (library) that assists in creating a test swf file, which runs all the tests in the browser. There's a few blog entries that give details on how to generate an XML file from the test swf, so breaking a build or reporting results is possible. I haven't bothered to go that far, but others have successfully blazed that trail.

In general, testing Flex is not great, but I rarely find myself concerned or unhappy. Here's a few reasons why.
  • There's little logic in my views anyway, the current app is Flex front end to Rails resources.
  • Anything that's annoying, I move out of my way by generating with some ruby scripts (for example: creating the file that lists all the test cases). -- more on this later
  • Anonymous objects in flex have been 'good enough' stubs
If you only want a high level opinion -- it's good enough, but not ideal or ground breaking.

I'm not a fan of manually adding tests to my test suite, but just like Paul Gross, we dynamically create ours. So it's not a problem. We didn't stop there though, we also generate the package and class name; therefore, our test classes stay pretty clean. The code below is an example of an entire test file (the ones that we work with).

public function testLimitSortOrderCriteriaXml():void {
var smartListCriteria:* = new SmartListCriteria();
smartListCriteria.addLimitCriteria({order:"an order"});
assertEquals("an order", smartListCriteria.toXml().sort.order);
}

public function testLimitSortFieldCriteriaXml():void {
var smartListCriteria:* = new SmartListCriteria();
smartListCriteria.addLimitCriteria({field:"a field"});
assertEquals("a field", smartListCriteria.toXml().sort.field);
}

Those methods wouldn't do much in isolation, but if you generate the surrounding (tedious) code, they work quite well. Here's the rake task that I use.

desc "Generate test files"
task :generate_tests do
tests = Dir[File.dirname(__FILE__) + "/../../test/flex/tests/**/*Test.as"].each do |file_path|
file_path = File.expand_path(file_path)
package = File.basename(File.dirname(file_path))
emit_file_path = (File.dirname(file_path) + "/" + File.basename(file_path, ".as") + "Emit.as").gsub(/tests/,"generated_tests")
File.open(emit_file_path, "w") do |file|
file << <<-eot
package generated_tests.#{package} {

import flexunit.framework.TestCase;
import flexunit.framework.TestSuite;
import uk.co.company.utils.*;
import uk.co.company.smartLists.*;
import mx.controls.*;

public class #{File.basename(emit_file_path,".as")} extends TestCase {

#{File.readlines(file_path)}
}
}
eot

end
end
end

As you can see, I generate test files and which are later used as source for the test swf. It's a few extra steps, but I never see those steps, I just write my test methods and run rake test:flex when I want to execute the tests. The one catch is that I can't run individual tests, which is terrible.

There are no mocking frameworks that I've seen, so that's kind of a bummer. Though, in practice I haven't found myself reaching for a mock anyway, but that will probably depend on your style. Again, I don't have much behavior in my views, so I don't need rich testing tools. Of course, I'd love to have them if they were available, but I don't find their loss to be a deal breaker. I do reach for stubs fairly often, but anonymous classes seem to work fine for that scenario.

There is one huge win that I'll conclude with. When using HTML & Javascript you have to test in browser to ensure that it works. This isn't true of Flex since there aren't browser issues. So, I can test my application using FlexUnit, and be confident that it just works. Removing the need for a (often slow and fragile) Selenium suite is almost enough to make me ditch HTML & Javascript forever.

6 comments:

  1. Thanks for these posts. Always interesting to read how folks do things elsewhere.

    ReplyDelete
  2. Jay, have you looked at http://funfx.rubyforge.org/ ? It's written by one of my BEKK colleagues - Peter Motzfeldt, and seems to be getting some traction....

    ReplyDelete
  3. Its nice to see someone finally blogging about flex testing. I feel like this is top 3 in my flex annoyances list.

    At work I've built a somewhat unsophisticated flex mocking/stubbing framework. Lets you do things like:

    var x:XStub = new XStub();
    x.stubs('method').withParams(1, 2).andReturn('value');

    and
    var x:XStub = new XStub();
    verify(x.shouldReceive('y').andReturn('z'))

    The whole thing is pretty useful since we generally type the arguments that we pass into things, so 9/10 times we can't just pass an Object in. The ugly thing is that you end up having to make XStub be an actual class that's defined with things like:

    class XStub extends X {
    include '../../......./Stubbable.as'
    public override function x(param:Object):Object { this.invokeStub('x', param); }
    }
    I did a lot of searching around for awhile trying to avoid having to do these sorts of things, but I'm pretty confident now that ActionScript 3 makes it impossible to write a real RSpec/mocha quality stubbing library, without forcing you to jump through hoops to use it.

    But then again, this's all a problem caused by falling into the static typing trap.

    Just needs a whitty name and maybe I can get it on github (then maybe someone can clean up some of the loose ends).

    ReplyDelete
  4. Anonymous8:16 AM

    You might be interested in:
    - mock-as3: http://thefoundry.anywebcam.com/index.php/actionscript/mock-as3-released-a-mock-object-library-for-actionscript-3/
    This is fine for simple mocks. For more advanced ones, you could start writing a library with the help of http://eval.hurlant.com/.

    - Dynamic creation of TestSuites AND filtering unit tests: http://www.adobe.com/cfusion/communityengine/index.cfm?event=showdetails&postId=7223&productId=2&loc=en_US

    ReplyDelete
  5. Anonymous7:35 AM

    Hi Marc,

    Thanks for the tip on mock-as3. I'll check it out.

    As for using Antennae, I avoid Ant and XML in general as much as possible. I strongly prefer the rake tasks I've created. Generating test suites is a small price to pay to be rid of XML based build scripts.

    Cheers, Jay

    ReplyDelete
  6. If you like easy way of mocking, maybe mockito for flex could work for you.
    http://bitbucket.org/loomis/mockito-flex

    ReplyDelete

Note: Only a member of this blog may post a comment.