Wednesday, July 09, 2008

Coding is dead, long live unit testing.

There are IDEs that can generate unit tests for your class once you've written the basic class stub. Or, they'll generate your class stub for you from the unit test code.

But a unit test suite should be written to cover all of the requirements in the requirements specification. It should be the contract of what is being developed.

So, if you've gone to all this effort to create a fully specified unit test suite, why should you then double your effort to write some more code to fulfil these requirements?

Couldn't something do this for you?

Like, a genetic software generation algorithm?

2 comments:

Dan said...

Hi Ian, interesting blog, just found it via Digg and your post on variable naming.

I think what you are suggesting here with genetic programming (http://www.genetic-programming.org/) is certainly feasible in some respects. However, GP requires huge processing resources for non-trivial problems because of the size of the search space. So there could be some practical limitations (for now at least).

Another stumbling block is that evolutionary computation requires fitness functions that are non-binary. That is to say that for a given requirement, we need to be able to measure how well a particular randomly evolved program satisfies that requirement. It is not sufficient to simply determine "yes this works" or "no this doesn't work", we need to be able to compare imperfect solutions and determine which one is closer to the desired goal.

I guess this means that we would need unit tests that don't just pass or fail but instead return a correctness score.

Fleejay said...

Thanks for the interesting comments.

In the case of a large unit test suite being created for a method, the fitness function could simply be the number of tests that passed.

I agree, that this won't work when there are only a small number of tests.

And of course, it won't work if there are tests that contradict one another.

A unit test that has a measure of how correct something is would be a radical shift in the way people write unit tests, but would be a viable approach.