All the Cases Covered

David‘s application has loads of unit tests. Many of the unit tests even go so far as to exhaustively test every combination of parameters. So seeing something like this is pretty common:

[Test]
[TestCase(false, false, false, false, false)]
[TestCase(false, false, false, false, true)]
[TestCase(false, false, false, true, false)]
[TestCase(false, false, false, true, true)]
[TestCase(false, false, true, false, false)]
[TestCase(false, false, true, false, true)]
[TestCase(false, false, true, true, false)]
[TestCase(false, false, true, true, true)]
[TestCase(false, true, false, false, false)]
[TestCase(false, true, false, false, true)]
[TestCase(false, true, false, true, false)]
[TestCase(false, true, false, true, true)]
[TestCase(false, true, true, false, false)]
[TestCase(false, true, true, false, true)]
[TestCase(false, true, true, true, false)]
[TestCase(false, true, true, true, true)]
[TestCase(true, false, false, false, false)]
[TestCase(true, false, false, false, true)]
[TestCase(true, false, false, true, false)]
[TestCase(true, false, false, true, true)]
[TestCase(true, false, true, false, false)]
[TestCase(true, false, true, false, true)]
[TestCase(true, false, true, true, false)]
[TestCase(true, false, true, true, true)]
[TestCase(true, true, false, false, false)]
[TestCase(true, true, false, false, true)]
[TestCase(true, true, false, true, false)]
[TestCase(true, true, false, true, true)]
[TestCase(true, true, true, false, false)]
[TestCase(true, true, true, false, true)]
[TestCase(true, true, true, true, false)]
[TestCase(true, true, true, true, true)]
public void UpdateClientSettingsTest(bool canCreateBeneficiary, 
	bool canCreatePayment, bool canCreateDeal, 
	bool canEditPlan, bool isPayrollEnabled) 
{

}

What a nice thorough test! Every possible case is tested! Despite the name “Update”, it inserts a pile of permissions records.

There are just a few problems with this test. The first is that the test isn’t set up to use a mock or in-memory database or anything that makes it easy to clean up: it uses a real test database. It also doesn’t do any cleanup between test cases. The test only passes because it ignores failures on inserts.

The other problem with the test is that the method it’s testing only writes to the database. It contains no logic. It simply takes six parameters (the five passed to this test, and a user ID), and constructs an INSERT statement to execute. The exhaustiveness of the test proves nothing, because all our code does is insert a record. Because the test is unsanitary, other tests confirm that the permissions settings behave as expected, but that’s more by accident than intent. It also means that, for this test suite, order matters– on a fresh database, if this test doesn’t run first, other tests will fail.

But hey, what a nice thorough unit test. It covers every possible case!