Just a Thought...

I’ve been putting off upgrading to JUnit 4 for a while. After all, just how much does it really buy you? It turns out, that JUnit 4 grants you a number of advantages that weren’t available with JUnit 3. One of those features is currently in an experimental phase: Theories. Theories let you specify a bunch of data points, which can be applied to each theory in your class. For example, if you want to test some boundary conditions on a class, you can make it happen easily like this:

@RunWith(Theories.class)
public class VerifyMyAlgorithm {
    @DataPoint
    public static int limLow = 0;

    @DataPoint
    public static int limHigh = 3000;

    @DataPoint
    public static int belowLimLow = limLow - limHigh;

    @DataPoint
    public static int aboveLimHigh = limHigh * 2;

    @DataPoint
    public static int median = (limHigh - limLow) / 2;

    @Theory
    public void verifyTwoParameters(int first, int second) {
        assertInRange(limLow, limHigh, MyMath.algorithm(first,second));
    }
}

I wanted to keep it simple just so you can get the basic idea. Essentially the @RunWith() annotation changes the basic behavior of your test class. It enables the use of the following annotations: @DataPoint, @DataPoints (more on this later), and @Theory. The data points you specified are collected and matched together so that each unique combination of datapoints is applied to the parameters in your theory. If your theory takes one parameter, the theory is run once per data point. If your theory takes two parameters it is run with every unique combination (15 times in this case).

But wait, there’s more! Sometimes we want to generate our datapoints with code. For example, we may need prime numbers up two four significant digits, or we need to initialize our data. In order to do that, we can use the @DataPoints annotation. The return type of your static method will be an array of datapoints. I could rewrite the example above like this:

@RunWith(Theories.class)
public class VerifyMyAlgorithm {
    @DataPoints
    public static int[] attemptLimits() {
        return new int[] {0,3000,-3000,6000,1500};
    }

    @Theory
    public void verifyTwoParameters(int first, int second) {
        assertInRange(limLow, limHigh, MyMath.algorithm(first,second));
    }
}

The DataPoints functionality in concert with the theories are the foundation of what is ncessary to automatically test the contracts of each service. I’ve thrown together a rudimentary class scanner that uses reflection to iterate over classes in the classpath to determine if they match the criteria for the service. For example, you can look at all classes that implement an interface, or extend a base class, or even have an annotation. From that list of classes we can test the contracts of the implementations. The nice aspect of this approach over creating a base test class and extend it manually for each implementation we write, is that it automatically finds the new implementations for you. I have yet to release the class scanner, but you can implement your own pretty well. Here is an example of how it would be used:

@RunWith(Theories.class)
public class EnforceLocakableContracts {
    @DataPoints
    public static Lockable[] collectImplementations() {
        Collection<Class<?>> klasses = new ClassCollector()
                .assignableTo(Locakable.class).recurse().collect();

       Lockable[] implementations = new Lockable[klasses.size()];
       int i = 0;
       for (Class<?> klass : klasses) {
           implementation[i] = klass.newInstance();
           i++;
       }

       return implementations;
    }

    @Theory
    public void lockShouldApplyToOneUser(Lockable lockable) {
        User one = TestSupport.createUser("one");
        User two = TestSupport.createUser("two");

        assertFalse(lockable.isLocked());

        lockable.acquireLock(one);

        assertFalse(lockable.canAccess(two);
    }

    @Theory
    public void lockableShouldBeAccessibleByLocker(Lockable lockable) {
        User one = TestSupport.createUser("test");

        assertFalse(lockable.isLocked());

        lockable.acquerLock(one);

        assertTrue(lockable.canAccess(one);
    }
}

So on and so forth. Just add a new Theory for each aspect of the implementing class you want to test. With this approach, testing implmentations of an interface is essentially future proofinf yourself against lazy programmers. There are some aspects that this approach doesn’t handle well just yet, such as complex setup for each object.

As of JUnit 4.7 there are a few problems, but they are not insurmountable:

• You see one pass/fail per theory (not per implementation)
• Failures do not show what implementation failed (fixed in the stacktraces provided by JUnit 4.8.1)
• The first failure kills future tests. You may have errors in multiple implementations but you won’t be able to find it until you fix the first implementation.

The best thing is to upgrade to JUnit 4.8.1, which is still not available in Maven repositories yet. However, you can also use the System.out approach to find out what the last thing tested was.

There is a feature request for theories to allow you to run them discretely. I.e. each run gets displayed in your test report individually with the parameters supplied in the test name. That will address the shortcomings and make this an even better approach.