Now we get feedback in the form of markers when a test fails. Which tests should we run? There is a strong tendency to be miserly running tests. This seems to be a legacy from the days of limited computing cycles. Our philosophy is to always run all the available tests. When performance makes this totally impossible, we find a way to write more granular tests that provide nearly the same feedback with far fewer CPU cycles. In theory we always run all tests, but we pragmatically allow reality to diverge from this theory on occasion.

Following this theory, we should run all the tests in a project whenever the project changes. How do we find all the tests in a project? In this chapter, we'll learn how to

This search functionality doesn't seem to fit nicely with any of our existing classes, so we'll create a new object, the TestSearcher:

public class TestSearcher {
}

The external API for the searcher is to pass it a project and receive in return an array of ITypes, representing the test classes in the project. We are ignoring test suites and other implementors of the Test interface for the moment.

The test for the searcher, SearchTest, relies on a TestProject being created in setUp() and disposed in tearDown():

Example . org.eclipse.contribution.junit.test/SearchTest

private TestProject testProject;
private IPackageFragment pack;
protected void setUp() throws Exception {
  testProject= new TestProject();
  testProject.addJar("org.junit", "junit.jar");
  pack= testProject.createPackage("pack1");
}
protected void tearDown() throws Exception {
  testProject.dispose();
}

The test adds three types to the project:

Example . org.eclipse.contribution.junit.test/SearchTest

public void testSearch() throws Exception {
  testProject.createType(pack, "AbstractTest.java",
    "public abstract class AbstractTest " +
    " extends junit.framework.TestCase { }");
  testProject.createType(pack, "NotATest.java",
    "public class NotATest { }");
  IType type= testProject.createType(pack, "ATest.java",
    "public class ATest extends junit.framework.TestCase { }");
  TestSearcher searcher= new TestSearcher();
  IType[] tests=searcher.findAll(testProject.getJavaProject(),
    null);
  assertEquals(1, tests.length);
  assertEquals(type, tests[0]);
}

Notice that we've added a parameter to TestSearcher.findAll(). For operations that are likely to run for a long time, Eclipse adds a Collecting Parameter,^[1] IProgressMonitor. We'll be calling methods that invite participation in reporting progress, so we want to give our clients the same opportunity. For testing purposes, we just pass in a null. We could also pass in an instance of NullProgressMonitor.^[2]

public IType[] findAll(IJavaProject project, IProgressMonitor pm)
    throws JavaModelException {
  IType[] candidates= getAllTestCaseSubclasses(project, pm);
  return collectTestsInProject(candidates, project);
}

We'll use a couple of helper methods to first get all visible subclasses of test case, then filter out the ones that don't actually live in this project. Notice that the API is written to return a typed array and not a List. It's easier for a client to deal with a typed array than a generic list, particularly if the array is only accessed and not modified. Typed arrays also eliminate the possibility of ClassCastExceptions.

We fetch all the subclasses of TestCase by first finding the IType representing the base type. If it doesn't exist in this project, there can be no test cases, so we return an empty array. From the type, we create its hierarchy. From the hierarchy, we fetch an array of all the subclasses.

private IType[] getAllTestCaseSubclasses(IJavaProject project,
    IProgressMonitor pm) throws JavaModelException {
  IType testCase= project.findType("junit.framework.TestCase");
  if (testCase == null)
    return new IType[0];
  ITypeHierarchy hierarchy= testCase.newTypeHierarchy(project, pm);
  return hierarchy.getAllSubtypes(testCase);
}

Why do we have to go through the intermediate ITypeHierarchy? Why can't we get all subclasses from the IType? Type hierarchy computation is expensive, particularly finding all subtypes. You have to know the entire state of the world to compute the subtypes correctly. Most users of IType don't need this information. By separating the ITypeHierachy, most users of IType don't have to pay the performance penalty. Notice that IType.newTypeHierarchy() takes an IProgressMonitor as a parameter, a hint that it can take a long time. If you only need the supertypes, use newSupertypeHierarchy(), don't create the whole hierarchy.

Now we have candidate types, but some of the test case classes may live in other projects or external libraries. We need to filter these out before returning the results.

private IType[] collectTestsInProject(IType[] candidates,
    IJavaProject project) {
  List result= new ArrayList();
  for (int i= 0; i < candidates.length; i++) {
    try {
      if (isTestInProject(candidates[i], project))
        result.add(candidates[i]);
    } catch (JavaModelException e) {
      // Fall through
    }
  }
  return (IType[]) result.toArray(new IType[result.size()]);
}

If we encounter a problem while evaluating a particular type, we'll just ignore that type and move on to the next one.

How do we decide if an IType is in a given project? First, we exclude all types that are external, contained in an external JAR and therefore have no underlying resource. We also exclude types whose “home” project is different than the project we're searching. Finally, we exclude abstract classes.

private boolean isTestInProject(IType type, IJavaProject project)
    throws JavaModelException {
  IResource resource= type.getUnderlyingResource();
  if (resource == null)
    return false;
  if (! resource.getProject().equals(project.getProject()))
    return false;
  return ! Flags.isAbstract(type.getFlags());
}

Notice that you don't query an IType directly for its modifiers, type.is-Abstract(). Instead, you ask for its flags and you call a helper class to decode the flags for you. This is consistent with the style introduced in the Java reflection API.

Next we'll see how to participate in the build process, so we can invoke the tests we've found. Summarizing this chapter, we