Note: I'm using the term feature in the sense of map feature, not in the sense of application feature or implemented requirement.
A simple pattern for such tests is
public class FeatureTest
{
@Test
public void testAllFeatures()
{
for (Feature feature : findAllFeatures())
{
testOneFeature(feature);
}
}
private void testOneFeature(Feature feature)
{
// some logic with one or more JUnit assertions
}
}
Obviously, this naive approach has the following drawbacks:
- The test fails and terminates on the first incorrect feature. The remaining features will not be tested.
- All features get tested sequentially. This may take awfully long for a large database.
@RunWith(Parameterized.class)
public class FeatureTest
{
// This is the parameter for each instance of the test.
private Feature feature;
public FeatureTest(Feature feature)
{
this.feature = feature;
}
@Parameters
public static Collection<Object[]> getParameters()
{
List<Feature> features = findAllFeatures();
List<Object> parameters = new ArrayList<Object[]>(features.size());
for (Feature feature : features)
{
parameters.add(new Object[] { feature };
}
return parameters;
}
@Test
public void testOneFeature()
{
// assertions acting on the feature member
}
}
This solves the first problem: Each feature gets tested in its own test instance. Now if there is a large number of features or if each individual test is very expensive, we would like to run the test instances in parallel, using a thread pool, or maybe even a grid of multiple computers.
Browsing through the JUnit sources, I found a surprisingly easy way of parallelizing the tests with a thread pool, simply by using a custom runner:
@RunWith(Parallelized.class)
public class FeatureTest
{
// same class body as above
}
All you need is a simple extension of the Parameterized runner:
public class Parallelized extends Parameterized
{
private static class ThreadPoolScheduler implements RunnerScheduler
{
private ExecutorService executor;
public ThreadPoolScheduler()
{
String threads = System.getProperty("junit.parallel.threads", "16");
int numThreads = Integer.parseInt(threads);
executor = Executors.newFixedThreadPool(numThreads);
}
@Override
public void finished()
{
executor.shutdown();
try
{
executor.awaitTermination(10, TimeUnit.MINUTES);
}
catch (InterruptedException exc)
{
throw new RuntimeException(exc);
}
}
@Override
public void schedule(Runnable childStatement)
{
executor.submit(childStatement);
}
}
public Parallelized(Class klass) throws Throwable
{
super(klass);
setScheduler(new ThreadPoolScheduler());
}
}
The
RunnerScheduler interface is fairly new in JUnit and marked as experimental. I discovered it in the current version JUnit 4.8.1 and found it missing in JUnit 4.4.0 which we have been using so far. RunnerScheduler is also available in JUnit 4.7.0, but I did not check if this is the earliest version.
