Thursday, May 10, 2007

Things I've learned on my new project and why VSTS unit testing tools suck

I recently started working with a new team and like any time you start something new there is a rush of learning that occurs. I would like to share some of what I've observer/learned while working with this team over the last few weeks. These are my opinions and subject to change...

1.) Visual Studio 2005 Unit Test (MS TEST) Runner is awful!

Don't get me wrong, I'm glad that Microsoft has a unit testing framework integrated with the IDE (as long as your using the "ultra" Team Suite edition). For most Microsoft devs, if it isn't baked into the IDE, they won't use or acknowledge the value of the technology.

Back to what makes the baked in tools so bad:

a.) All these views and the one that I want doesn't exist!

Do you need the Test Manager, the Test View, the Test Result or the Test Result Detail window? Don't ask me, because I haven't figured out a good way to use them...yet.

If you don't know the difference between the views, here are some thumbnails. Click on the image to see a larger version. You tell me which one I should use (and you can't say TestDriven.NET).

Worst of all the "xUnit" view doesn't exist. A simple tree view in the left pane with a console window on the right to view exceptions and output values.

I'm using James Kocacs' VstsUnit Plugin for ReSharper. (A huge Thank You to James) It gives me what I need.

b.) Unit test architecture is slow

They must be copying the test assemblies to another location and starting another process. It is slow! I'm a firm believer that a unit test should execute in less than a 100th of a second (no dependencies on external systems - Yes, the file system is an external system.), and my unit tests still take multiple seconds to respond with the first result.

c.) Code Gen'd Accessor class feels dirty

If you haven't worked with the the VSTS testing tools, then you don't know about the class that Visual Studio generates for you. This code-gened class wraps the class you are testing and thru the magic of reflection, allow you access to private fields, properties and methods.

I both love and hate this. I love the convenience of it, but hate that it allows badly designed classes to be unit tested. The first sign that I've created a poorly designed class is that I can't unit test it....Actually that doesn't happen because I create a unit test first, but I do run into this when I'm trying to get legacy code into a test harness.

2.) Resharper = Love

I've been away from her too long, but I rekindled the flame with Resharper. Don't tell my wife!

3.) Developers need tools

Speaking for Resharper, the first question I will ask when I'm considering a project is, "Can I install software on my machine?" When you leave a company that won't allow you to install a font, you start to appreciate the little things that make you productive.

4.) Dual Screen Monitors = More Love

Now I have to find dual screen wallpapers