So we had our second meeting on Monday, this is a brief summary of what was discussed.
- Details on how Assert works, the fact that the methods are static.
- Testing Exceptions: This led to a very interesting discussion on how most frameworks handle this.
We talked about the Assert.Throws from NUnit and how its really nice to be able to do something like this
Assert.Throws<NullReferenceException>(user.WillThrowAnnNullExecption);
or even something like this
var e= Assert.Throws<NullReferenceException>(e.WillThrowANullExecption);
Assert.AreEqual(“Object reference not set to an instance of an object.”, e.Message);
Ok the message in this case is not very valuable but you can see how you can get very specific, the fact that the exception is returned is really valuable. By the way I just tried this and it is true 
- Test failures could lead us to write more tests. We discussed the ways we handle this: some people use todo lists ( as per TDD by Example by Kent Beck), some people write test that are not implemented, some people use a combination of both.
- Liked the fact that it mentioned that tests should be independent of each other at this early stage of the book.
-Test Initialization Setup and TearDown as well as FixtureSetUp and FixtureTearDown and how they work.
I thought this was a good place to mention this as it took me a while to know that they existed, also mentions the advantage of having one place to call the constructor of the class under test.
David showed us something a bit smarter, he uses a factory that creates a valid object with the required properties set up using lambdas. This is good because you can see what is that you need in your test and its in one line and still have all the constructor usage in one place.
- Ignore attribute. After discussing, this seemed like a bad idea unless you were very disciplined.
- Category. The consensus was that we really didn’t think it was a great idea in general to use this feature, because you might not run all tests. Might be a “test smell”, good to know is there tho.
- Finally the book mentions Indirectly testing state, I think, as a way to give way to the next chapter.
Other things:
We keep mentioning these three books at all times ( no specific order)
Working Effectively with Legacy Code
We were also talking about mbUnit versions and actually today i saw this blog post about mbUnit being in version 3.1.
Jacob mentioned that the existence of a test support for Silverlight, which is great, however he also mentioned that is possible to mark something as a bug so the test will pass, this didn’t seem so good.
Did I forget something?
3 Comments
Interesting stuff ye are covering…
Although I wasn’t at the meeting I have info that ye may/may not find useful:
We predominately use NUnit as our test runner. (We are looking at others as well)
To test exceptions we:
1. Flag the test with [Test, ExpectedException(typeof(SomeException))]
2. If exception is thrown in SUT do nothing. If throwing of an exception from the SUTs external dependency, use a mocking framework to mock up the throwing of an exception t5o see how the SUT handles it
We use TTD so we have lots of tests that are flagged with:
[Test, ExpectedException(typeof(NotImplementedException))]
These tests numbers should decrease as the project matures over time…
In fact we also have other exceptions in tests such as
[Test, ExpectedException(typeof(NotSupportedException))]
We also have pre-post condition exceptions for DBC verification
Finally, I would strongly recommend this book: xUnit Test Patterns
http://www.amazon.co.uk/xUnit-Test-Patterns-Refactoring-Signature/dp/0131495054
Cheers for the comment Billy,
The way you describe testing for exceptions is the same way it’s decribed in the book, however we were thinking that this way it could fail to test properly when you have very general Exceptions such as NullReferenceException because there might be ocasions where the exception happens but for the wrong reasons. ok you can add a Message but then you are relaying on strings, what happens if you are runnign the tests in a machine in a different language? the test will fail for the wrong reasons.
I might try your idea of using [Test, ExpectedException(typeof(NotImplementedException))] as I tend to create a few not implemented tests but have to switch to a list when there are too many. Thanks for the tip
Cheers
Andrea
I see where you are coming from in relation to adding strings – however they can be problematic like you described.
In my opinion if a test for an SUT expects a NullReferenceException is expected to be thrown, your test should describe exactly how the SUT is expected to behave – not just the fact that a NullReferenceException is expected to be thrown – AS IT COULD BE THROWN FROM ANYWHERE!!!.
In other words this really is a test smell!
By using a mocking framework you can describe exactly the path the code should take with the SUT.
# This would be in SetUp
Dependency1 d1 = this.c_mockRepository.StrictMock();
Dependency1 d2 = this.c_mockRepository.StrictMock();
Dependency1 d3 = this.c_mockRepository.StrictMock();
[Test, ExpectedException(typeof(NullReferenceException())]
public void Test_Demoing_Power_Of_Mocking__To_Exactly_Define_Expected_Test_Behaviour()
{
using(this.c_mockRepository.Record()
{
Expect.Call(d1.Method1(null)).Return(true);
LastCall.IgnoreArguments();
Expect.Call(d2.Method2(null,null).Return(MyFactory.Method2_GoodReturnObject());
LastCall.IgnoreArguments();
Expect.Call(d3.Method3(null).Throw(new NullReferenceException()));
LastCall.IgnoreArguments();
}
using (this.c_mockRepository.Playback())
{
new SystemUnderTestObj(d1,d2,d3).Process();
}
}
For tests where no dependencies are involved and logic is quite small e.g. testing DBC checks in constructors, having e.g. [Test, ExpectedException(typeof(NullReferenceException())] is acceptable by itself IMO.