Some Test Driven Development observations

Testing code is hard. There is some material available, but most of the examples are trivial. When you actually sit down to do strictly Test Driven Development (TDD) the complications appear quickly.

The first tripping point for me was cascading failures. Simple changes to one class often caused several test failures. Sometimes the fix was a one-liner, but often it was not. I began to find the solution when I first read Martin's paper on Mocks and Stubs. After I read that paper I began to use Mocks religiously, but that didn't really solve the problem. Using an excessive amount of mocks is just as fragile as using the actual implementations.

To me, the next distinct progression of my testing skills was following the one assertion per test guideline. Using only one assertion per test allowed me to write very focused tests that generally had less dependencies. By removing dependencies my tests became more robust.

Understanding Dependency Injection was the next major step in improving my testing skills. By using Dependency Injection I was able to further abstract my dependencies to ensure robust tests.

At this point I felt my tests had improved significantly; however, my test suite was becoming a bit of a burden to execute. The major problem was the excessive network traffic between my test suite and the database. To me, long build times are a serious problem because it represents lost developer time, several times a day. I quickly understood the distinction between Unit tests and Functional (or Integration) tests.

The next logical step was to improve test performance by not crossing boundaries while unit testing. Utilizing Dependency Injection I began to mock the data access code in my unit tests which significantly improved test processing time.

Further increasing the robustness of my tests was achieved by testing one concrete class at a time. Again, Dependency Injection helped provide the solution. By using Dependency Injection the code I wrote was significantly decoupled compared to the previous code I was writing. Injecting mocks or stubs for dependencies of my class under test did increase the robustness of the tests; however, I still had the issue of dealing with fragile mock set up code.

A common practice is to specify method names using strings when setting up expectations of a mock. This is obviously fragile when performing constant refactoring. However, using stubs always seemed like more trouble than it was worth because it required implementing an entire interface when often I only cared about a few methods. With no clear answer on the 'best' choice I used both excessively in different places to determine which approach I preferred. In the end, I found that neither should be treated as the silver bullet. In fact, using both mocks and stubs it's possible to make very clear what your test is accomplishing.

Another gray area in testing concerns what visibility to give your classes members. When I first started out testing I believed that exposing the private members for testing was acceptable. However, I quickly learned this approach was naive and a much better solution is to simply test through the public interface. Also, if a method is marked as private, but I feel it needs to be tested I likely wouldn't hesitate to simply make it public.

The reason I believed I needed to access private methods was because I was using a Code Coverage tool that told me my code was only about 80% covered. However, the answer wasn't that I needed to devise a solution to reach 100%, it was that I needed someone to tell me that 80% or higher is good enough. Should you strive for more than 80%? Sure, but don't make any compromises to get there. You will find the return on investment is generally not near as valuable.

That's the majority of my previously documented lessons learned concerning testing. I hope this entry and my previous entries can provide some guidance and help you avoid some common stumbling blocks. In part 2 of this entry I'll document some observations that I haven't previously written about.