Maintainability and Expect Literals

Recently, Stephen Schaub asked the following on the wewut group:

Several of the unit test examples in the book verify the construction of both HTML and plain text strings. Jay recommends using literal strings in the assertions. However, this strikes me as not a particularly maintainable approach. If the requirements regarding the formatting of these strings changes (a very likely scenario), every single test that verifies one of these strings using a literal must be updated. Combined with the advice that each test should check only one thing, this leads to a large number of extremely brittle tests.

Am I missing something here? I can appreciate the reasons Jay recommends using literals in the tests. However, it seems that we pay a high maintainability price in exchange for the improved readability.

I responded to Stephen; however, I've seen similar questions asked a few times. Below are my extended thoughts regarding literals as expected values.

In general, given the option of having many similar strings (or any literal) vs a helper function, I would always prefer the literal. When a test is failing I only care about that single failing test. If I have to look at the helper function I no longer have the luxury of staying focused on the single test; now I need to consider what the helper function is giving me and what it's giving all other callers. Suddenly the scope of my work has shifted from one test to all of the tests coupled by this helper function. If this helper function wasn't written by me, this expansion in scope wasn't even my decision, it was forced upon me by the helper function creator. In the best case the helper function could return a single, constant string. The scope expansion becomes even worse when the helper function contains code branches.

As for alternatives, my solution would depend on the problem. If the strings were fairly consistent, I would likely simply duplicate everything knowing that any formatting changes can likely be addressed using a bulk edit via find and replace. If the strings were not consistent, I would look at breaking up the methods in a way that would allow me to verify the code branches using as little duplication as possible, e.g. if I wanted to test a string that dynamically changed based on a few variables, I would look to test those variables independently, and then only have a few tests for the formatting.

A concrete example will likely help here. Say I'm writing a trading system and I need to display messages such as

"paid 10 on 15 APPL. $7 Commission. spent: $157"
"paid 1 on 15 VTI. Commission free. spent: $15"
"sold 15 APPL at 20. $7 Commission. collected: $293"
"sold 15 VTI at 2. Commission free. collected: $30"

There's quite a bit of variation in those messages. You could have 1 function that creates the entire string:
confirmMsg(side, size, px, ticker)

However, I think you'd end up with quite a few verbose tests. Given this problem, I would look to break down those strings into smaller, more focused functions, for example:

describeOrder(side, size, px, ticker)
describeCommission(ticker)
describeTotal(side, size, px, ticker)

Now that you've broken down the function, you're free to test the code paths of the more focused functions, and the test for confirmMsg becomes trivial. Something along the lines of

assertEquals("paid 10 on 15 APPL",
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("sell 15 APPL at 10",
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"standard"}))

assertEquals("$7 Commission", 
  describeCommission({tickerName:"APPL",commission:"standard"}))
assertEquals("Commission free", 
  describeCommission({tickerName:"APPL",commission:"free"}))

assertEquals("spent: $157", 
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("collected: $143", 
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("spent: $150", 
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"free"}))
assertEquals("collected: $150", 
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"free"}))

assertEquals("order. commission. total", 
  confirmMsg("order", "commission", "total"))

I guess I could summarize it by saying, I should be able to easily find and replace my expected literals. If I cannot, then I have an opportunity to further break down a method and write more focused tests on the newly introduced, more granular tests.

ActionScript: Literal XML

One feature of ActionScript I really like is the ability to use XML within the language. In ActionScript there's no need to use strings to represent XML, you can use XML inline just as you would any other literal (numbers, strings, etc).

Recently I was working on some dynamic code that needed to update based on an XML response. We were just getting started on the card, so instead of worrying about the service, how it was triggered, etc, we added a button to the interface that called the response parsing method and passed in an XML literal as an argument. The code is similar to what's shown below.

parseResponse(<user><firstName>Jay</firstName></user>);

Sure, we had to change the code later and make a real service call, but this quick solution let us keep focusing on the task at hand instead of how to get the XML. In a language like Ruby we could have used a builder or just a string, and given such a small amount of XML it would have been fine. However, the actual XML we were working with was significantly larger than the example and would have been a decent mess of a multiline string or several builder calls. Being able to write XML natively was significantly easier.

The beauty of literal XML is the simplicity. I don't have to represent it in any way other than what it actually is -- XML.

If you aren't a fan of all the angle brackets, that's okay, ActionScript has you covered there also. You can add elements and attributes as you would expect to be able to if you prefer method calls.

var request:XML = <smart_list/>;
request.sort.order = "highest";
request.sort.field = "Average Position";
request.max_results = 10;
request.toXMLString(); // "<smart_list><sort><order>highest</order><field>Average Position</field></sort><max_results>10</max_results></smart_list>"

If you only ever use XML similar to the above syntax, then there may be little value in literal XML, but I don't think you're usage would be limited to the above syntax.

I write tests, a lot of them. I prefer to see the actual XML in my tests, instead of the builder versions of XML. I don't like angle brackets any more than the next programmer, but I strongly prefer testing with expected literals. I find my resulting tests to be more readable and reliable. If you've ever had a test fail because of whitespace issues in your XML, you should know what I mean by reliable.

Literal XML also ensures XML compliance at compile time and encourages IDEs to provide syntax highlighting. Two things that aren't essential, but are definitely nice to have.

ActionScript is the first language I've used with literal XML support, and I'm very happy with the experience. As programmers we spend a lot of time hiding other languages with object relational mappers (orm) and builders, but literal XML is a refreshing step in the other direction. The creators of ActionScript have embraced the fact that XML isn't going anywhere. They built first class support for XML in the language itself instead of hiding the problem with a framework, and that helps me significantly more than any orm or builder ever has.