Unit testing is defined as:
... a procedure used to verify that a particular module of source code is working properly. The idea about unit tests is to write test cases for all functions and methods so that whenever a change causes a regression, it can be quickly identified and fixed. Ideally, each test case is separate from the others ... [Wikipedia]
There are existing practices that I consider to be good guidelines concerning unit testing.
- William Caputo - Do Not Cross Boundaries. The central idea is to decouple the code from the resources. An added benefit is the reduced running time of the tests. The faster the tests are, the more often they are run.
- Dave Astels - One Assertion Per Test. While the idea is controversial it's also very thought provoking. If you follow this rule your tests become simple, expressive, and elegant. I don't follow this rule 100% of the time; however, I do believe it's a great guideline.
I follow another guideline that I haven't seen documented anywhere.
Only one concrete class (with behavior) should be used per test.The central reason for testing concrete classes individually is to promote durable tests. When several concrete classes are used in one test the test becomes brittle. Making a change to any of the coupled concrete classes can cause cascading test failures.
Both mocks and stubs can be used in place of concrete classes where necessary while testing an individual class. If you find your hierarchy too complicated to use mocks or stubs this is probably a sign that you need a simpler and less coupled hierarchy. Using Object Mother is a common alternative to refactoring to a better hierarchy. If you find yourself reaching for Object Mother, take the time to refactor instead.
I wish more TWorkers think about unit test in that way. Completely agree with your post.
ReplyDeleteI would disagree to your last principle - it solves one problem (cascade failures) and introduces another (brittle tests). Good OO designs typically have small but highly cohesive classes collaborating to achieve a purpose. Assuming such a design, using mocks means every time you refactor classes, you need to touch your unit tests in a big way. Not to mention the amount of fixture required to setup mocks for testing.
ReplyDeleteA better approach I have found is to write unit tests against components (I know you are shaking your head in disbelief :). This reduces test maintainability, as a component has a well-defined interface which changes less often than class interface.
Even with component tests, one still needs to mock resource layers (DB, messaging, et al) to make test runs fast.
-RR
When you have tightly coupled object graphs and are using mocks then you can run into this problem.
ReplyDeleteHowever, if you write smaller, more loosely coupled objects much of this pain can be removed. Additionally, if you use stubs instead of mocks your refactoring tool should change the stub for you.
The tests you are writing are not truly unit tests anyway. You are testing your code as several classes working together, this is a functional test.
You indeed have an interesting PoV.
ReplyDeleteI agree with you point on small loosely-coupled objects. But they are orthogonal. if classes are small and highly cohesive, then they have to delegate and collaborate to achieve a unit of functionality. In my experience this is often true for domain models Often quoted example is Order-LineItem-Product. Order.calculatePrice() would loop thru its items and call the item's calcPrice() which calls Product.getPrice(). Then if I requirements change such that the point-in-time price needs to be captured in the line-item, then I would change the LineItem's calcPrice(). If I were writing my unit test against Order, the tests require little to no change, but otherwise the mocks would have had to change to accomodate this.
Re, #2 (using stubs) - With stubs, structural refactorings would taken care of - I agree.
But the expectation setting code would have to change if object collaborations change. This is a very common refactoring. If using mocks or stubs, one still has to assert that the expected collaborations of an object actually happened.This is what causes test maintenance issues. Even with removing restrictions of order of calls, etc. the tests still seem brittle if you follow "refactor mercilessly".
Re #3 - Well, they are called unit tests, not class tests or object tests. They exercise a "unit" of functionality. In my case they are components. I prefer to call them unit tests because I'm still testing the component in isolation (no dependencies to resource layer or other components). My unit is just coarser than yours.
To me, functional tests are tests focussing on functionality - the unit tests I write may not have that focus - it is testing the component interface which may or may not be synonymous.
-RR
PS: BTW, your blog is indeed an interesting read - I came across it very recently. Keep up the good work!
RR,
ReplyDelete1. I actually expected more people to disagree. I was surprised when I didn't get criticism. I don't take it personally, I rather enjoy differences of opinions. I feel that different PoVs help me to grow.
2. Have you ever heard that XP is several practices that don't work as well unless you do as many as you can? I feel that this rule applies to my programming style also. I have seen the problem that you are commenting on. However, when I follow things such as constructor injection, law of demeter, behavior based testing, etc. I find that these things work very will with very fine grained unit tests. Also, the tests are durable even during merciless refactoring.
3. Having lived through the brittle tests I can totally understand your PoV. Unfortunately, unless we are staffed on a project together I'm likely never going to be able to demonstrate the effective style that fine grained unit tests require.
Thanks for the kind words. And, please comment whenever you disagree. We wouldn't want people to start believing that I know what I'm talking about. =)
Mock sucks - especially dont try to issolate 1 class at a time!
ReplyDeleteMock suck
RR - I totally agree with you. Unit tests for components is much better. Fewer mocks the better. then they are not brittle if you want to refactor.
ReplyDeleteThe unexpected consequences of the collaberation is what you want to tests. You also want to test from the clients point of view - I.e. test you commponent interfaces. If the component happens to take an interface as an input then sure use a Mock for that but otherwise you are not fully testing your code