Testing: Unit Test or Functional Test

Update at bottom.

A common question when adding a new class is: Do I need a Unit test or Functional test?
The answer: Yes.

I'm not sure why, but there seems to be a common misconception that if you have a Unit test a Functional test is not necessary. This simply is not true.

I believe it may stem from the days where we only had one test suite. In those days the entire suite was essentially a Functional test suite. However, we've learned many lessons since then, and some of those lessons have allowed us to create an additional test suite for testing the units of our system in isolation. And, while it's still true that sometimes you only need a Functional test, it's rarely true that you will only want a Unit test.

I love to talk about different strategies concerning Unit Testing. And, often during those discussions holes are pointed out where the unit tests will not fail despite non-functioning code. These holes appear at the class integration points, which are not exposed when testing classes in isolation. This is why creating Functional Tests is crucial to test driven development.

So, if we must write Functional tests anyway, what's the use in having Unit tests? I like to create Unit tests to verify edge cases and slightly different code paths. For example, imagine a phone number class that contains a method for returning formatted string representations of the phone number.

class PhoneNumber < ActiveRecord::Base
  def to_formatted_s type
    case type
      when :en then ...
      when :fr then ...
    end
  end
end

The tests for the to_formatted_s method can be Unit tests that do not hit the database. It's possible to test all the scenarios required by to_formatted_s without the overhead of hitting the database. Avoiding trips to the database wont make a big difference if your entire test suite is 50 tests; however, as test suites grow, often so does the time to execute them.

If I were testing the PhoneNumber class I would create Unit tests for each logical path through the to_formatted_s method and I would create one Functional test that tested the same method, but actually used an instance of the PhoneNumber class that interacted with the database. This strategy would give me many Unit tests that quickly test the method and one Functional test that runs slower, but ensures integration is valid. Working this way should allow me to create a large test suite without creating a long running test suite.

You know you are in a bad situation when you don't want to run the tests because they take too long to execute. If you find yourself in this position, creating a test suite that cuts out trips to external dependencies (such as dbs or web services) is probably a good choice. The trips to the external dependencies are still required, but can be limited by testing as much logic as possible in the Unit tests and only hitting the external dependencies in the Functional tests.

Update
I got this comment that I thought was valuable enough it was worth adding to the post.

Erock said...

Hmm, maybe I don't quite understand, but why are functional tests needed in this case? You've tested all cases in your Unit tests, but then decide that to feel comfortable with your tests, you MUST hit the DB? Wouldn't it be ideal not to hit the db at all? How is a functional test going to give you better results then isolated unit tests? I was interested in this post, but you left more to the imagination than your usual very well thought out posts.

Sorry, I should have been more specific. The method under test, to_formatted_s, is going to format the data from from the area_code, exchange, and station attributes. Those attributes come from the columns that are defined on the phone_numbers table. Therefore, since the to_formatted_s method depends on attributes that are created on the database side it's a good idea to functionally test that the data coming from those columns is formatted correctly. This can prevent you from changing a column name in the database and not having a test that fails because your unit tests stub all the column names.

However, if the attributes of the phone number class didn't come from the database, then I wouldn't necessarily need a functional test for the to_formatted_s method.

The more common case is where methods aren't so simple that they can execute in isolation. Often methods interact with other classes, and those classes should be mocked/stubbed in Unit tests. But those methods should also be tested in a Functional test while collaborating to ensure proper integration.