MarkUs Blog

MarkUs Developers Blog About Their Project

Archive for the ‘test taxonomy’ tag

Quick Word on Test Taxonomy

with one comment

One thing (among lots of others) I have learned about tests during last semester is that, depending on the environment (or the culture you’re developing in) tests get different names.

Here’s an example where Rails’ naming conventions differ from what I have learned in my QA course[1]:

Rails’ Unit Tests are not Unit Tests

Considering a unit test should test an object in complete isolation, it is a breach to that rule to have a “unit” test access the database. Rails’ unit tests are really functional tests where we make sure that the model behaves accordingly (for one) and that its mapping with the database is correct (for two). Another thing about unit tests is that they should be fast to run. Under 30 seconds is a generally good standard (varying depending on the environment). Let me guess, you need somewhat more than 30 seconds to run rake test:units on MarkUs. For example, on my machine I get a run time of approximately 31 seconds, which would be good, if it was not for the fact that it does not include the database setup time as well as (I believe) setup and teardown time between each test[2].

Side-note: I also get 1 error and 1 failure. It makes me want to emphasize that fast-to-run tests get ran more often. It the tests are not used, they loose their ability to help find bugs early, get deprecated and more of a burden that a great tool.

On the other hand. Every model object is so tightly coupled with ActiveRecord that one could argue that we do not really want to test our model classes without ActiveRecord participating.

Anyway, this is an example of what I was stating in the first paragraph. In this case, it is commonly accepted, in Rails’ culture, that, what I could refer to as “open box integration testing of models and ActiveRecord”[3] should be called “a unit test”. The later has some clear advantages over the former and the most important thing is to make sure that everyone is speaking the same language among the team.

Refactoring the test/ folder

As Adam told us during code sprint. It is a common practice, among Rails’ developer teams, to refactor the test folder on the very start of a new project. One way of doing this could give something like:

  • test/ remains test/
  • unit/ becomes model/
  • functional/ becomes controller/
  • unit/ gets added (and contains model unit tests that does not hit the database)
  • functional/ gets added (and contains selenium tests)
  • acceptance/ gets added (and contains cucumber tests)

Being loose

I have myself been pretty loose all these times I have reffered to Cucumber tests simply as acceptance tests. If you dig a little, you’ll find that I was more specifically talking about “user acceptance” tests.

Convention Over Elite Correctness

It’s written up there, but the key thing is to speak the same language among the team. You’ve experienced it, Rails is all about conventions. Keeping the usual setting may help newcomers to the project, but used Rails developers, to understand exactly what is supposed to go where (and what they can expect to find where) when it comes to test.

On the other hand, from what I can tell, newcomers to the project also are newcomers to Ruby and Rails, which, I believe, leaves the way open for a little structure refactoring.


  1. Which, in turn, was not proposing an absolute answer to “how a specific test should be named?” but rather focused on providing us with the necessary tools (read: knowledge) to efficiently and intelligently test the software we develop.
  2. Does anybody know?
  3. and I am no authority on the question. I also intentionally made it too long for its own good 😉

Written by gabrielrl

December 4th, 2009 at 5:03 pm

Posted in Uncategorized