Comments on: Book Club – Chapter 2 The Art of Unit Testing

By: Billy Stack

Billy Stack — Wed, 16 Sep 2009 15:57:25 +0000

I see where you are coming from in relation to adding strings – however they can be problematic like you described.

In my opinion if a test for an SUT expects a NullReferenceException is expected to be thrown, your test should describe exactly how the SUT is expected to behave – not just the fact that a NullReferenceException is expected to be thrown – AS IT COULD BE THROWN FROM ANYWHERE!!!.

In other words this really is a test smell!
By using a mocking framework you can describe exactly the path the code should take with the SUT.

# This would be in SetUp
Dependency1 d1 = this.c_mockRepository.StrictMock();
Dependency1 d2 = this.c_mockRepository.StrictMock();
Dependency1 d3 = this.c_mockRepository.StrictMock();

[Test, ExpectedException(typeof(NullReferenceException())]
public void Test_Demoing_Power_Of_Mocking__To_Exactly_Define_Expected_Test_Behaviour()
{
using(this.c_mockRepository.Record()
{
Expect.Call(d1.Method1(null)).Return(true);
LastCall.IgnoreArguments();

Expect.Call(d2.Method2(null,null).Return(MyFactory.Method2_GoodReturnObject());
LastCall.IgnoreArguments();

Expect.Call(d3.Method3(null).Throw(new NullReferenceException()));
LastCall.IgnoreArguments();
}
using (this.c_mockRepository.Playback())
{
new SystemUnderTestObj(d1,d2,d3).Process();
}
}

For tests where no dependencies are involved and logic is quite small e.g. testing DBC checks in constructors, having e.g. [Test, ExpectedException(typeof(NullReferenceException())] is acceptable by itself IMO.

By: roundcrisis

roundcrisis — Wed, 16 Sep 2009 13:23:31 +0000

Cheers for the comment Billy,
The way you describe testing for exceptions is the same way it’s decribed in the book, however we were thinking that this way it could fail to test properly when you have very general Exceptions such as NullReferenceException because there might be ocasions where the exception happens but for the wrong reasons. ok you can add a Message but then you are relaying on strings, what happens if you are runnign the tests in a machine in a different language? the test will fail for the wrong reasons.

I might try your idea of using [Test, ExpectedException(typeof(NotImplementedException))] as I tend to create a few not implemented tests but have to switch to a list when there are too many. Thanks for the tip

Cheers

Andrea

By: Billy Stack

Billy Stack — Wed, 16 Sep 2009 12:02:00 +0000

Interesting stuff ye are covering…

Although I wasn’t at the meeting I have info that ye may/may not find useful:

We predominately use NUnit as our test runner. (We are looking at others as well)
To test exceptions we:
1. Flag the test with [Test, ExpectedException(typeof(SomeException))]
2. If exception is thrown in SUT do nothing. If throwing of an exception from the SUTs external dependency, use a mocking framework to mock up the throwing of an exception t5o see how the SUT handles it

We use TTD so we have lots of tests that are flagged with:
[Test, ExpectedException(typeof(NotImplementedException))]
These tests numbers should decrease as the project matures over time…

In fact we also have other exceptions in tests such as
[Test, ExpectedException(typeof(NotSupportedException))]
We also have pre-post condition exceptions for DBC verification

Finally, I would strongly recommend this book: xUnit Test Patterns
http://www.amazon.co.uk/xUnit-Test-Patterns-Refactoring-Signature/dp/0131495054