Thursday, 25 September 2008

Rspec mocks too much

I've recently had to learn the other side of testing - ie rspec, due to my new contract having a test suite written in "the other testing suite" * ;)

Seems fairly straightforward so far, but there's one big worry I have: Rspec has a huge reliance upon mocks.

Now I completely understand (and endorse) the benefits of using mocks. It means you can independantly test a model/controller/whatever without it mattering if something happens to have broken in some other model/module/whatever. This is a Good Thing and allows for more accurate testing due to all the benefits of loose coupling. However, I get a bit worried if everything is mocked out and there are no integration-style tests at all.

For example, if I'm testing a view, and using rspec best practice I've mocked out all model objects and don't bother with the real thing. What happens if a model changes (say a column is removed that shows up on an index list)? The view specs all rely on a mock that explicitly states what is returned, so the view specs all still pass. But the index page will be broken for a user that actually uses the site, because the real object doesn't bahave that way anymore. The mocks and the model are out of synch. :P

In an infrequently-used area of the site, this sort of error may go undetected and slip through to release.

This worries me. I know I like to rely on my test suite to catch things like this wherever possible. Sure I generally do a manual click-through, but I know that this is unreliable especially WRT edge-cases.

So, am I needlessly worrying? Have I simply missed something about how rspec is supposed to be used? What's best practice in this situation?

*Which admittedly feels a lot like having to switch from vi to emacs... Sure I know that "the other one" works for lots of people, and I meant to get around to learning it someday...


Anonymous said...

As much I know about rspec, there are no default best practices yet. Try what makes sense and are useful for you and keep your mind open to accept new trends (what comes out from practice). For example, controller and especially view specs are shadowed by cucumber ( I have fall in love cucumber and selenium tests and doing all views

Taryn East said...

AFAIK there aren't any "written rules" for best practise - but these things tend to grow organically. What I was hoping for were to have a look at some specs that people would consider to cover the leading-edge of how rspec "should be used" - which is generally how best practise arises.

I did the same thing while learning test::unit and now have my own doc full of good practise (ie maybe not best, but still a good start).

I gathered these from reading blogposts, and writing my own tests and finding out neat tricks and when neat tricks fail.

I guess was hoping somebody else might have written a post like this about rspec ;)