tag:blogger.com,1999:blog-12467669.post116722141440620934..comments2023-04-29T07:23:25.825-04:00Comments on Jay Fields' Thoughts: Rails: Unit Testing ActiveRecord ValidationsJayhttp://www.blogger.com/profile/14491442812573747680noreply@blogger.comBlogger9125tag:blogger.com,1999:blog-12467669.post-10818710541779339242007-09-25T16:39:00.000-04:002007-09-25T16:39:00.000-04:00> -Would you prefer to test all the permutations o...> -Would you prefer to test all the permutations of valid and invalid situations or write one test and count on the framework to correctly implement the validation.<BR/><BR/>I think this point is interesting. Certainly we shouldn't be testing that the framework's validation mechanism works, but surely it's key here that the same validation mechanism works on external input (the attributes being passed), so to test this class properly we should be sure that, given the inputs we're likely to encounter, the validation enables our class to respond as expected.<BR/><BR/>Given the email address example, perhaps it's more useful to consider validating its form. Simply testing:<BR/><BR/> <B>Person.expects(:validates_format_of).with(:email_address, :with => EMAIL_REGEXP)</B><BR/><BR/>doesn't strike me as a good test, because we've more than likely just reused the same EMAIL_REGEXP as we wrote in the implementation. Surely this is a case where we'd want to supply a set of malformed strings and check that the validation (i.e. the regular expression that isn't part of the framework) we've declared actually behaves how we'd hoped?<BR/><BR/>I suppose putting it succinctly, it does seem less compelling to add many tests for very simple declarative validations, but I think that was more a reflection of the simple example than a general rule.Unknownhttps://www.blogger.com/profile/09709877938805757843noreply@blogger.comtag:blogger.com,1999:blog-12467669.post-58764435611270021712007-09-25T14:09:00.000-04:002007-09-25T14:09:00.000-04:00You're not really testing the framework in these c...You're not really testing the framework in these cases. You're testing that you're calling the framework in the correct way to accomplish what you intend to do.<BR/><BR/>The problem with the 2nd example is that you're assuming in the test that you know what the correct framework call is to accomplish what you intend to do. When I've tried mocking calls for similar tests, my first assumption has almost always been incorrect. The other problem is that your test says how you MUST write the code, when there may be other (better?) ways to accomplish the same thing.Craig Buchekhttps://www.blogger.com/profile/11598114439645095559noreply@blogger.comtag:blogger.com,1999:blog-12467669.post-78114098164647034012007-07-26T15:38:00.000-04:002007-07-26T15:38:00.000-04:00I also think I prefer the first approach that test...I also think I prefer the first approach that tests behaviour. There are advantages to testing framework functionality as well. It makes sure that the framework behaves as you expect and that it continues to do so after future upgrades.Peter Marklundhttps://www.blogger.com/profile/11921485901587530645noreply@blogger.comtag:blogger.com,1999:blog-12467669.post-20096080022692498812007-01-14T18:52:00.000-05:002007-01-14T18:52:00.000-05:00The problem with testing the implementation (like ...The problem with testing the implementation (like expecting some validation declaration method to be called), IMHO, that it works only in simple cases. If you have some conditional validation (like "require this field not to be empty only if some other field has particular value") you will need to fine tune the expected arguments for validation declaration call or otherwise you will be testing some other validation, not the one you actually wanted to test. So, my opinion is to have all those complex scenarios explicitly tested instead of relying on assumption, that having certain validation setup will cover all of them.<br /><br />Then, you are bound to certain validation implementation. If you would change validation implementation later (e.g. for efficiency reason) and drop <i>validates_*</i> methods in favor of custom <i>validate</i> method you would require rewriting tests.<br /><br />Then, you say that testing valid? method require model to be valid. I tend to use following pattern:<br /><br />class UserTest < Test::Unit::TestCase<br /> test 'test data should be valid' do<br /> assert_valid new_user<br /> end<br /><br /> test 'should require name' do<br /> user = new_user :name => nil<br /> assert_attribute_invalid user, :name<br /> end<br /><br /> test 'should require email' do<br /> user = new_user :email => nil<br /> assert_attribute_invalid user, :email<br /> end<br /><br /> protected<br /><br /> def new_user(params = {})<br /> User.new({ :name => 'Jay', :email => 'jay@example.com' }.merge(params))<br /> end<br />end<br /><br /><br />So, here I have test to ensure that helper method creates valid user object that I could use as a base for my validation tests.<br /><br />Also, to follow 'one assertion per test' principle, I've added an <i>assert_attribute_invalid</i> method which actually does two tests behind the scene:<br /><br />def assert_attribute_invalid object, attribute<br /> assert ! object.valid?<br /> assert object.errors.on(attribute)<br />endAnonymoushttps://www.blogger.com/profile/00272049513122973998noreply@blogger.comtag:blogger.com,1999:blog-12467669.post-1167743113521346512007-01-02T08:05:00.000-05:002007-01-02T08:05:00.000-05:00Very interesting. I've seen (read 'written') some ...Very interesting. I've seen (read 'written') some pretty hairy validation specs that attempt to cover all the variations. The second approach would really clean them up.<BR/><BR/>I see Luke's point. It does feel "implementy". But I think you can make a fair argument that testing that the validation gets set by mocking the interface to AR is exactly what we would do if the interface to AR looked a little different.<BR/><BR/>Imagine if AR provided some sort of config class. Then you might do something like this:<BR/><BR/>specify "should tell config to require email" do<BR/> @config.should_receive(:required_fields).with(@model_class, :email)<BR/>end<BR/><BR/>Seems completely natural given that design.<BR/><BR/>Thanks Jay - a real thought-provoker.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-12467669.post-1167404957762035672006-12-29T10:09:00.000-05:002006-12-29T10:09:00.000-05:00luke: The problem with the statement "testing beha...luke: The problem with the statement "testing behaviour not implementation details" is that it is absolutely correct, in isolation. However, when you start adding variables to the context the lines aren't nearly as distinct between right and wrong.<BR/><BR/>Here's a few thoughts along those lines<BR/>-Do you write get and set tests for accessors to test their behavior?<BR/>-Testing a model correctly validates an email address requires a round trip to the database to get the email address column, or you can mock the columns, but that requires some set up. So, is it worth the db connection or the additional set up?<BR/>-Also, what behavior do you test? A validation doesn't just set the behavior of the errors collection, it also sets the behavior of the valid? method. Assuming you have multiple validations, in order for you to test that the valid? method is returning correctly for each error you will need to set up valid data for every validation except the one that you are testing.<BR/>-Do you actually need a test that tests behavior that is not yours? Yes, you could write the validation yourself, but when would you ever do that for a presence of validation? So, are you writing a test for a situation that you would never encounter? And, since you likely are, is it worth the trip to the db and the previously mentioned set up.<BR/>-The 'test behaviour not implementation' mantra was born in languages where declaritive programming is not possible. Are you willing to blindly apply to declaritive programming also?<BR/>-Would you prefer to test all the permutations of valid and invalid situations or write one test and count on the framework to correctly implement the validation.<BR/><BR/>There are more examples, but I think that list is enough to show that we are far from a clear answer on this topic, at this point.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-12467669.post-1167368101492335222006-12-28T23:55:00.000-05:002006-12-28T23:55:00.000-05:00If I write a test for say, "User must have an emai...If I write a test for say, "User must have an email address" I might write a test like:<BR/><BR/>http://pastie.caboo.se/30056<BR/><BR/>(actually I'd write it in a more BDD-oriented style with RSpec but thats besides the point).<BR/><BR/>I'm expressing a requirement that the user should be invalid when it doesn't have an email address. This test doesn't test implementation, nor does it test framework functionality. It simply expresses a business requirement without caring how that test is made to pass. And of course we could implement that using our own home-grown solution but following the mantra of "do the simplest thing that works", with Rails, this means using a validation macro.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-12467669.post-1167319207924824782006-12-28T10:20:00.000-05:002006-12-28T10:20:00.000-05:00A problem with the first example is that you are t...A problem with the first example is that you are testing the behavior of the validation; however, you didn't implement the behavior of the implementation. In essence, you are testing that the framework correctly built the validation for the field you specified. And, I try to avoid testing code I'm not responsible for (i.e. frameworks) as often as possible.<BR/><BR/>That said, testing the implementation isn't a great plan either. But, is it the best of bad options?Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-12467669.post-1167267826625035152006-12-27T20:03:00.000-05:002006-12-27T20:03:00.000-05:00I'd be inclined to avoid the second one completely...I'd be inclined to avoid the second one completely - you should be testing behaviour not implementation details.Anonymousnoreply@blogger.com