Maintainability and Expect Literals

Recently, Stephen Schaub asked the following on the wewut group:

Several of the unit test examples in the book verify the construction of both HTML and plain text strings. Jay recommends using literal strings in the assertions. However, this strikes me as not a particularly maintainable approach. If the requirements regarding the formatting of these strings changes (a very likely scenario), every single test that verifies one of these strings using a literal must be updated. Combined with the advice that each test should check only one thing, this leads to a large number of extremely brittle tests.

Am I missing something here? I can appreciate the reasons Jay recommends using literals in the tests. However, it seems that we pay a high maintainability price in exchange for the improved readability.

I responded to Stephen; however, I've seen similar questions asked a few times. Below are my extended thoughts regarding literals as expected values.

In general, given the option of having many similar strings (or any literal) vs a helper function, I would always prefer the literal. When a test is failing I only care about that single failing test. If I have to look at the helper function I no longer have the luxury of staying focused on the single test; now I need to consider what the helper function is giving me and what it's giving all other callers. Suddenly the scope of my work has shifted from one test to all of the tests coupled by this helper function. If this helper function wasn't written by me, this expansion in scope wasn't even my decision, it was forced upon me by the helper function creator. In the best case the helper function could return a single, constant string. The scope expansion becomes even worse when the helper function contains code branches.

As for alternatives, my solution would depend on the problem. If the strings were fairly consistent, I would likely simply duplicate everything knowing that any formatting changes can likely be addressed using a bulk edit via find and replace. If the strings were not consistent, I would look at breaking up the methods in a way that would allow me to verify the code branches using as little duplication as possible, e.g. if I wanted to test a string that dynamically changed based on a few variables, I would look to test those variables independently, and then only have a few tests for the formatting.

A concrete example will likely help here. Say I'm writing a trading system and I need to display messages such as

"paid 10 on 15 APPL. $7 Commission. spent: $157"
"paid 1 on 15 VTI. Commission free. spent: $15"
"sold 15 APPL at 20. $7 Commission. collected: $293"
"sold 15 VTI at 2. Commission free. collected: $30"

There's quite a bit of variation in those messages. You could have 1 function that creates the entire string:
confirmMsg(side, size, px, ticker)

However, I think you'd end up with quite a few verbose tests. Given this problem, I would look to break down those strings into smaller, more focused functions, for example:

describeOrder(side, size, px, ticker)
describeCommission(ticker)
describeTotal(side, size, px, ticker)

Now that you've broken down the function, you're free to test the code paths of the more focused functions, and the test for confirmMsg becomes trivial. Something along the lines of

assertEquals("paid 10 on 15 APPL",
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("sell 15 APPL at 10",
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"standard"}))

assertEquals("$7 Commission", 
  describeCommission({tickerName:"APPL",commission:"standard"}))
assertEquals("Commission free", 
  describeCommission({tickerName:"APPL",commission:"free"}))

assertEquals("spent: $157", 
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("collected: $143", 
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"standard"}))
assertEquals("spent: $150", 
  describeOrder("buy", 10, 15, {tickerName:"APPL",commission:"free"}))
assertEquals("collected: $150", 
  describeOrder("sell", 10, 15, {tickerName:"APPL",commission:"free"}))

assertEquals("order. commission. total", 
  confirmMsg("order", "commission", "total"))

I guess I could summarize it by saying, I should be able to easily find and replace my expected literals. If I cannot, then I have an opportunity to further break down a method and write more focused tests on the newly introduced, more granular tests.