Exploration of Unit Test Principle

By Fang Yuan (Mingchu)

Preface

Unit test is an important part of software development. A good unit test can help us find problems earlier and guarantee stable system operation. Unit test is also good documentation, from which we can understand the author's design intent for classes. Code refactoring is also inseparable from the unit test. Rich unit test cases will fill us with confidence when we refactor codes. Although the unit test is important, the operating and configuration principle has not been very clear. Thus, I explored the unit test principle and recorded it here.

What Happens When Running Unit Tests in IDEA?

First, let's look at what IDEA helps do when we run a unit case directly through IDEA:

It compiles the project source code and test source code and outputs the results to the target directory.

It runs the com.intellij.rt.junit.JUnitStarter by a java command, and the parameter specifies the version of JUunit and the name of the unit test case.

java com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit4 fung.MyTest,test

Here, we focus on the code of JUnitStarter. This class is in the junit-rt.jar plug-in package provided by IDEA. The specific directory is /Applications/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit-rt.jar. This package can be introduced into our own engineering projects to facilitate reading source codes:

Main Function of JUnitStarter:

public static void main(String[] args) {
    List< String> argList = new ArrayList(Arrays.asList(args));
    ArrayList< String> listeners = new ArrayList();
    String[] name = new String[1];
    String agentName = processParameters(argList, listeners, name);
    if (!"com.intellij.junit5.JUnit5IdeaTestRunner".equals(agentName) && !canWorkWithJUnitVersion(System.err, agentName)) {
        System.exit(-3);
    }
    
    if (!checkVersion(args, System.err)) {
        System.exit(-3);
    }

    String[] array = (String[])argList.toArray(new String[0]);
    int exitCode = prepareStreamsAndStart(array, agentName, listeners, name[0]);
    System.exit(exitCode);
}

Here are two core approaches:

...
// Process parameters. It is for determining which version of the JUnit framework is used and populating listeners based on the input parameters.
String agentName = processParameters(argList, listeners, name);
...
// Start the test.
int exitCode = prepareStreamsAndStart(array, agentName, listeners, name[0]);
...

Next, look at the sequence diagram when the prepareStreamsAndStart method is running. Here, JUnit4 is used as an example:

After IDEA confirms the framework version to be started, it creates an instance of IdeaTestRunner< ?> through the fully qualified name of the class and the reflection mechanism. Here, JUnit4 is used as an example. IDEA instantiates the object of com.intellij.junit4.JUnit4IdeaTestRunner and calls its startRunnerWithArgs method. In this method, IDEA builds the org.junit.runner.Request by the buildRequest method, obtains the org.junit.runner.Description by the getDescription method, creates the org.junit.runner.JUnitCore instance, and calls its run method.

In short, IDEA will eventually use the capabilities of the Junit4 framework to start and run the unit test case, so it is necessary to do some in-depth exploration of the source code of the Junit4 framework.

Exploration of Junit4 Source Code

Junit is a unit test framework written in Java language, which has been widely used in the industry. Its developers are Kent Beck (author of Refactoring: Improving the Design of Existing Code and Test-Driven Development) and Erich Gamma (author of Design Patterns and the father of Eclipse). Junit4 was released in 2006. Although Junit4 was out of date, it is necessary to explore the design concepts and ideas in it.

First of all, let's start with a simple unit test case:

public class MyTest {

    public static void main(String[] args) {
        JUnitCore runner = new JUnitCore();
        Request request = Request.aClass(MyTest.class);
        Result result = runner.run(request.getRunner());
        System.out.println(JSON.toJSONString(result));
    }

    @Test
    public void test1() {
        System.out.println("test1");
    }

    @Test
    public void test2() {
        System.out.println("test2");
    }

    @Test
    public void test3() {
        System.out.println("test3");
    }

}

Instead of using the IDEA plug-in to start a unit test, we directly start it through the Main function, whose core code is listed below:

public static void main(String[] args) {
  // 1. Create an instance of JUnitCore
  JUnitCore runner = new JUnitCore();
  // 2. Build the Request by performing a unit test on the Class object of class
  Request request = Request.aClass(MyTest.class);
  // 3. Run the unit test
  Result result = runner.run(request.getRunner());
  // 4. Print the result
  System.out.println(JSON.toJSONString(result));
}

Focus on runner.run(request.getRunner()). First, look at the code of the run function:

It can be seen that the type of test process that is run depends on the incoming Runner instance; different Runners determine different running processes. It can be guessed roughly by the name of the implementation class: JUnit4ClassRunner should be the basic test process of JUnit4, MockitoJUnitRunner possibly introduces the ability of Mockito, SpringJUnit4ClassRunner might have some connection with the Spring, and it may start the Spring container.

Now, let's look at the code of request.getRunner() in runner.run(request.getRunner()):

public Runner getRunner() {
  if (runner == null) {
    synchronized (runnerLock) {
      if (runner == null) {
        runner = new AllDefaultPossibilitiesBuilder(canUseSuiteMethod).safeRunnerForClass(fTestClass);
      }
    }
  }
  return runner;
}

public Runner safeRunnerForClass(Class< ?> testClass) {
  try {
    return runnerForClass(testClass);
  } catch (Throwable e) {
    return new ErrorReportingRunner(testClass, e);
  }
}

public Runner runnerForClass(Class< ?> testClass) throws Throwable {
  List< RunnerBuilder> builders = Arrays.asList(
    ignoredBuilder(),
    annotatedBuilder(),
    suiteMethodBuilder(),
    junit3Builder(),
    junit4Builder()
  );

  for (RunnerBuilder each : builders) {
    Runner runner = each.safeRunnerForClass(testClass);
    if (runner != null) {
      return runner;
    }
  }
  return null;
}

It can be seen that Runner is selected based on the information of the incoming testClass. The rules are listed below:

If the parsing fails, return ErrorReportingRunner
If there is an @Ignore annotation on the test class, return IgnoredClassRunner
If there is an @RunWith annotation on the test class, instantiate a Runner with the value of @RunWith and return it
If canUseSuiteMethod=true, return SuiteMethod inherited from JUnit38ClassRunner, being an earlier version of JUnit
If the JUnit version is earlier than 4, return JUnit38ClassRunner
If none of the above is satisfied, return BlockJUnit4ClassRunner, which represents a standard JUnit4 test model.

In the simple example we gave earlier, BlockJUnit4ClassRunner is returned, so take BlockJUnit4ClassRunner as an example to see how its run method is executed.

First, we will go to the run method in its parent class, ParentRunner:

@Override
public void run(final RunNotifier notifier) {
  EachTestNotifier testNotifier = new EachTestNotifier(notifier,
                                                       getDescription());
  try {
    Statement statement = classBlock(notifier);
    statement.evaluate();
  } catch (AssumptionViolatedException e) {
    testNotifier.addFailedAssumption(e);
  } catch (StoppedByUserException e) {
    throw e;
  } catch (Throwable e) {
    testNotifier.addFailure(e);
  }
}

It is necessary to detail the Statement here. The official explanation is:

Represents one or more actions to be taken at runtime in the course of running a JUnit test suite

The Statement can be simply understood as the encapsulation and abstraction of executable methods. For example, RunBefores is a Statement that encapsulates all methods annotated with @BeforeClass. These methods will be executed before running the use case of the unit case class. After running, RunBefores will also run the subsequent Statement through next.evaluate(). Here are some common Statements:

RunBefores runs the encapsulated method in befores first (usually annotated with @BeforeClass or @Before) and then runs next.evaluate()
RunAfters runs next.evaluate() first and then runs the method encapsulated in the afters (usually annotated with @AfterClass or @After)
InvokeMethod directly runs the method encapsulated in testMethod

It can be seen that the entire unit test operation process is the operation process of a series of Statements. Taking the previous MyTest as an example, the execution process of its Statement can be roughly summarized below:

The last question is: how does the actually tested method run? The answer is the reflection call. The core code is listed below:

@Override
public void evaluate() throws Throwable {
  testMethod.invokeExplosively(target);
}

public Object invokeExplosively(final Object target, final Object... params)
  throws Throwable {
  return new ReflectiveCallable() {
    @Override
    protected Object runReflectiveCall() throws Throwable {
      return method.invoke(target, params);
    }
  }.run();
}

So far, the execution process of a unit test case of standard Junit4 has been analyzed. How can we perform a unit test on Spring?

Exploration of Spring Unit Test

Let's start with a simple example:

@RunWith(SpringRunner.class)
@ContextConfiguration(locations = { "/spring/spring-mybeans.xml" })
public class SpringRunnerTest {

    @Autowired
    private MyTestBean myTestBean;

    @Test
    public void test() {
        myTestBean.test();
    }

}

Here's a rough overview of what happened when running a unit test. First,@RunWith annotates the test class, so the Junit framework will first create an instance of SpringRunner with SpringRunnerTest.class as the parameter and then call the run method of SpringRunner to run the test. In this method, the Spring container will be started to load the Bean configuration file specified by @ContextConfiguration annotation. Then, it will process the @Autowired annotation to inject myTestBean into the instance of SpringRunnerTest and run the test().

In short, start the Spring container through SpringRunner and run the test method. Next, explore the process of SpringRunner starting the Spring container:

public final class SpringRunner extends SpringJUnit4ClassRunner {

  public SpringRunner(Class< ?> clazz) throws InitializationError {
    super(clazz);
  }

}
public class SpringJUnit4ClassRunner extends BlockJUnit4ClassRunner {
  ...
}

SpringRunner and SpringJUnit4ClassRunner are equivalent. It can be considered that SpringRunner is an alias of SpringJUnit4ClassRunner. Here, we focus on the implementation of the SpringJUnit4ClassRunner class.

SpringJUnit4ClassRunner inherits BlockJUnit4ClassRunner. BlockJUnit4ClassRunner has been analyzed earlier. It runs a standard JUnit4 test model. SpringJUnit4ClassRunner has made some extensions on this basis. The extensions mainly include the following:

The constructor is extended to create an additional TestContextManager instance.
The createTest() method is extended to call the prepareTestInstance method of TestContextManager.
beforeClass is extended. The beforeTestClass method of TestContextManager is called before the method annotated with @BeforeClass is executed.
before is extended, the beforeTestMethod method of TestContextManager is called before the method annotated with @Before is executed.
afterClass is extended, the afterTestClass method of TestContextManager is called after the method annotated with @AfterClass is executed.
after is extended. The after method of TestContextManager is called after the method annotated with @After is executed.

TestContextManager is the core class of the Spring test framework. The official explanation is: TestContextManager is the main entry point into the Spring TestContext Framework. Specifically, a TestContextManager is responsible for managing a single TestContext.TestContextManager manages TestContext, which is a re-encapsulation of ApplicationContext. TestContext can be understood as a Spring container added with test-related functions. TestContextManager also manages TestExecutionListeners, where the observer mode is used to provide the ability to listen to key nodes (such as beforeClass, afterClass, etc.) during the test process.

Therefore, after studying the code of relevant implementation classes: TestContextManager, TestContext, and TestExecutionListeners, it is not difficult to find the startup secret of the Spring container during testing. The key code is listed below:

public class DefaultTestContext implements TestContext {
  ...
  public ApplicationContext getApplicationContext() {
    ApplicationContext context = this.cacheAwareContextLoaderDelegate.loadContext(this.mergedContextConfiguration);
    if (context instanceof ConfigurableApplicationContext) {
      @SuppressWarnings("resource")
      ConfigurableApplicationContext cac = (ConfigurableApplicationContext) context;
      Assert.state(cac.isActive(), () ->
                   "The ApplicationContext loaded for [" + this.mergedContextConfiguration +
                   "] is not active. This may be due to one of the following reasons: " +
                   "1) the context was closed programmatically by user code; " +
                   "2) the context was closed during parallel test execution either " +
                   "according to @DirtiesContext semantics or due to automatic eviction " +
                   "from the ContextCache due to a maximum cache size policy.");
    }
    return context;
  }
  ...
}

In the getApplicationContext method of DefaultTestContext, the loadContext of cacheAwareContextLoaderDelegate is called, and the refresh method of the Context is called to build the Spring container context. The sequence diagram is listed below:

Where is the getApplicationContext method called?

As mentioned earlier, TestContextManager extends the createTest() method, and its prepareTestInstance method is also called.

public void prepareTestInstance(Object testInstance) throws Exception {
  if (logger.isTraceEnabled()) {
    logger.trace("prepareTestInstance(): instance [" + testInstance + "]");
  }
  getTestContext().updateState(testInstance, null, null);

  for (TestExecutionListener testExecutionListener : getTestExecutionListeners()) {
    try {
      testExecutionListener.prepareTestInstance(getTestContext());
    }
    catch (Throwable ex) {
      if (logger.isErrorEnabled()) {
        logger.error("Caught exception while allowing TestExecutionListener [" + testExecutionListener +
                     "] to prepare test instance [" + testInstance + "]", ex);
      }
      ReflectionUtils.rethrowException(ex);
    }
  }
}

All prepareTestInstance methods of TestExecutionListener are called in the prepareTestInstance method, where a listener called DependencyInjectionTestExecutionListener will call the getApplicationContext method of TestContext.

public void prepareTestInstance(TestContext testContext) throws Exception {
  if (logger.isDebugEnabled()) {
    logger.debug("Performing dependency injection for test context [" + testContext + "].");
  }
  injectDependencies(testContext);
}

protected void injectDependencies(TestContext testContext) throws Exception {
   Object bean = testContext.getTestInstance();
   Class< ?> clazz = testContext.getTestClass();
   
   // Call the getApplicationContext method of TestContext to build the Spring container.
   AutowireCapableBeanFactory beanFactory = testContext.getApplicationContext().getAutowireCapableBeanFactory();
   
   beanFactory.autowireBeanProperties(bean, AutowireCapableBeanFactory.AUTOWIRE_NO, false);
   beanFactory.initializeBean(bean, clazz.getName() + AutowireCapableBeanFactory.ORIGINAL_INSTANCE_SUFFIX);
   testContext.removeAttribute(REINJECT_DEPENDENCIES_ATTRIBUTE);
}

There is one final question. How was the DependencyInjectionTestExecutionListener added? The answer is spring.factories.

At this point, the startup process of the Spring unit test has been explored and understood. Next, look at the SpringBoot.

Exploration of SpringBoot Unit Test

A Simple Example of SpringBoot Unit Test:

@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
public class MySpringBootTest {

    @Autowired
    private MyTestBean myTestBean;

    @Test
    public void test() {
        myTestBean.test();
    }

}

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@BootstrapWith(SpringBootTestContextBootstrapper.class)
public @interface SpringBootTest {
  ...
}

As a rough note, the test is started through the run method of SpringRunner, which starts the Spring container, while @SpringBootTest provides the startup class. At the same time, the SpringBootTestContextBootstrapper class provided by @BootstrapWith enriches the capabilities of TestContext, making it support some features of SpringBoot. Here, we will focus on @BootstrapWith and SpringBootTestContextBootstrapper.

When I introduced TestContextManager earlier, I didn't talk about its constructor and the instantiation process of TestContext. Here is their explanation:

public TestContextManager(Class< ?> testClass) {
  this(BootstrapUtils.resolveTestContextBootstrapper(BootstrapUtils.createBootstrapContext(testClass)));
}

public TestContextManager(TestContextBootstrapper testContextBootstrapper) {
  this.testContext = testContextBootstrapper.buildTestContext();
  registerTestExecutionListeners(testContextBootstrapper.getTestExecutionListeners());
}

public abstract class AbstractTestContextBootstrapper implements TestContextBootstrapper {  
  ...
  public TestContext buildTestContext() {
    return new DefaultTestContext(getBootstrapContext().getTestClass(), buildMergedContextConfiguration(),
        getCacheAwareContextLoaderDelegate());
  }
  ...
}

Three parameters are required to build the DefaultTestContext:

testClass, the metadata of the class being tested
MergedContextConfiguration encapsulates the test container-related annotations declared on the test class (such as @ContextConfiguration, @ActiveProfiles, @TestPropertySource)
CacheAwareContextLoaderDelegate (used for loading or closing containers)

What should we do when we need to extend the functions of TestContext or when we don't want to use DefaultTestContext? The simplest way is to write a new class implementation TestContextBootstrapper interface and override the buildTestContext() method. Then, how do we tell the test framework to use the new implementation class? That's where @BootstrapWith comes in.

Here's a look at the code of BootstrapUtils.resolveTestContextBootstrapper:

static TestContextBootstrapper resolveTestContextBootstrapper(BootstrapContext bootstrapContext) {
  Class< ?> testClass = bootstrapContext.getTestClass();

  Class< ?> clazz = null;
  try {
    clazz = resolveExplicitTestContextBootstrapper(testClass);
    if (clazz == null) {
      clazz = resolveDefaultTestContextBootstrapper(testClass);
    }
    if (logger.isDebugEnabled()) {
      logger.debug(String.format("Instantiating TestContextBootstrapper for test class [%s] from class [%s]",
                                 testClass.getName(), clazz.getName()));
    }
    TestContextBootstrapper testContextBootstrapper =
      BeanUtils.instantiateClass(clazz, TestContextBootstrapper.class);
    testContextBootstrapper.setBootstrapContext(bootstrapContext);
    return testContextBootstrapper;
  }
  ...
}

private static Class< ?> resolveExplicitTestContextBootstrapper(Class< ?> testClass) {
  Set< BootstrapWith> annotations = AnnotatedElementUtils.findAllMergedAnnotations(testClass, BootstrapWith.class);
  if (annotations.isEmpty()) {
    return null;
  }
  if (annotations.size() == 1) {
    return annotations.iterator().next().value();
  }

  // Obtain the value of the @BootstrapWith annotation
  BootstrapWith bootstrapWith = testClass.getDeclaredAnnotation(BootstrapWith.class);
  if (bootstrapWith != null) {
    return bootstrapWith.value();
  }

  throw new IllegalStateException(String.format(
    "Configuration error: found multiple declarations of @BootstrapWith for test class [%s]: %s",
    testClass.getName(), annotations));
}

Here, the customized TestContextBootstrapper will be instantiated with the value of the @BootstrapWith annotation to provide a customized TestContext.

SpringBootTestContextBootstrapper is the implementation class of TestContextBootstrapper, which extends the ability to create the TestContext by indirectly inheriting the AbstractTestContextBootstrapper class. These extensions include:

ContextLoader is replaced with SpringBootContextLoader.
DefaultTestExecutionListenersPostProcessor is added to enhance TestExecutionListener.
Processing of webApplicationType has been added

Next, look at the relevant codes of the SpringBootContextLoader:

public class SpringBootContextLoader extends AbstractContextLoader {

  @Override
  public ApplicationContext loadContext(MergedContextConfiguration config)
      throws Exception {
    Class< ?>[] configClasses = config.getClasses();
    String[] configLocations = config.getLocations();
    Assert.state(
        !ObjectUtils.isEmpty(configClasses)
            || !ObjectUtils.isEmpty(configLocations),
        () -> "No configuration classes "
            + "or locations found in @SpringApplicationConfiguration. "
            + "For default configuration detection to work you need "
            + "Spring 4.0.3 or better (found " + SpringVersion.getVersion()
            + ").");
    SpringApplication application = getSpringApplication();
    // Set mainApplicationClass
    application.setMainApplicationClass(config.getTestClass());
    // Set primarySources
    application.addPrimarySources(Arrays.asList(configClasses));
    // Add configLocations
    application.getSources().addAll(Arrays.asList(configLocations));
    // Obtain environment
    ConfigurableEnvironment environment = getEnvironment();
    if (!ObjectUtils.isEmpty(config.getActiveProfiles())) {
      setActiveProfiles(environment, config.getActiveProfiles());
    }
    ResourceLoader resourceLoader = (application.getResourceLoader() != null)
        ? application.getResourceLoader()
        : new DefaultResourceLoader(getClass().getClassLoader());
    TestPropertySourceUtils.addPropertiesFilesToEnvironment(environment,
        resourceLoader, config.getPropertySourceLocations());
    TestPropertySourceUtils.addInlinedPropertiesToEnvironment(environment,
        getInlinedProperties(config));
    application.setEnvironment(environment);
    // Obtain and set initializers
    List< ApplicationContextInitializer< ?>> initializers = getInitializers(config,
        application);
    if (config instanceof WebMergedContextConfiguration) {
      application.setWebApplicationType(WebApplicationType.SERVLET);
      if (!isEmbeddedWebEnvironment(config)) {
        new WebConfigurer().configure(config, application, initializers);
      }
    }
    else if (config instanceof ReactiveWebMergedContextConfiguration) {
      application.setWebApplicationType(WebApplicationType.REACTIVE);
      if (!isEmbeddedWebEnvironment(config)) {
        new ReactiveWebConfigurer().configure(application);
      }
    }
    else {
      application.setWebApplicationType(WebApplicationType.NONE);
    }
    application.setInitializers(initializers);
    // Run the SpringBoot application
    return application.run();
  }

}

It can be seen that SpringApplication is built, mainApplicationClass, primarySources, and initializers are set, and the SpringBoot application is started through the application.run().

Therefore, the startup process of the SpringBoot unit test is clear. Next, let's look at how the Maven plug-in runs the unit test.

How Maven Plug-In Runs the Unit Test

We know that maven uses a series of plug-ins to help us complete the construction, testing, packaging, deployment, and other actions during project development. When running maven clean test commands in the Console, maven will run the following goals in sequence:

maven-clean-plugin:2.5:clean: It is used to clean up the target directory.
maven-resources-plugin:2.6:resources: Move the resource files in the main project directory to the classes directory in the target directory
maven-compiler-plugin:3.1:compile: Compile the java source code in the main project directory into the bytecode and move it to the classes directory in the target directory.
maven-resources-plugin:2.6:testResources: Move the resource files in the test project directory to the test-classes directory in the target directory
maven-compiler-plugin:3.1:testCompile: Compile the java source code in the test project directory into the bytecode and move it to the classes directory in the target directory
maven-surefire-plugin:2.12.4:test: Run a unit test

Let's look at the code of maven-surefire-plugin. First, introduce the following maven-surefire-plugin and surefire-junit4 packages to facilitate our review of the code:

<dependency>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-surefire-plugin</artifactId>
  <version>2.9</version>
</dependency>

<dependency>
  <groupId>org.apache.maven.surefire</groupId>
  <artifactId>surefire-junit4</artifactId>
  <version>3.0.0-M7</version></dependency>

The core code is in org.apache.maven.plugin.surefire.AbstractSurefireMojo#execute. Those interested can take a look. In short, the JUnit4Provider object will be instantiated by reflection with the information in JUnit4ProviderInfo and then its invoke method will be called. During the method modification, Runner will eventually be instantiated, and its run method will be called. The core code is listed below:

private static void execute( Class< ?> testClass, Notifier notifier, Filter filter )
{
  final int classModifiers = testClass.getModifiers();
  if ( !isAbstract( classModifiers ) && !isInterface( classModifiers ) )
  {
    Request request = aClass( testClass );
    if ( filter != null )
    {
      request = request.filterWith( filter );
    }
    Runner runner = request.getRunner();
    if ( countTestsInRunner( runner.getDescription() ) != 0 )
    {
      runner.run( notifier );
    }
  }
}

Summary

This is the end of the relevant principle of the unit test. Let's review it.

When we run a unit test directly through IDEA, the main method of JUnitStarter will work as an entry point and call Junit to run the unit test.
Junit4 abstracts the method annotated with @Before, @Test, and @After into statements. The whole operation process of the unit test is a series of operation processes of Statements. The method calls are implemented by reflection.
With the @RunWith(SpringRunner.class) annotation, the test framework will run the run method of the SpringRunner instance, create the TestContext through the TestContextManager, and start the Spring container. SpringRunner and SpringJUnit4ClassRunner are equivalent.
With the help of annotations: @SpringBootTest and @BootstrapWith(SpringBootTestContextBootstrapper.class), the test framework enhances the TestContext through the SpringBootTestContextBootstrapper to achieve the purpose of starting SpringBoot applications.
Maven starts a unit test by running maven-surefire-plugin:2.12.4:test, whose core is to call the code of the JUnit framework through JUnit4Provider.

Community

Exploration of Unit Test Principle

Preface

What Happens When Running Unit Tests in IDEA?

Exploration of Junit4 Source Code

Exploration of Spring Unit Test

Exploration of SpringBoot Unit Test

How Maven Plug-In Runs the Unit Test

Summary

Read previous post:

Read next post:

Alibaba Cloud Community

You may also like

Comments

Alibaba Cloud Community

Related Products

Penetration Test

Storage Capacity Unit