Exploring the Working Principle of Unit Testing

Related Tags：EMAS Mobile Testing
2. Java Unit Testing Skill - PowerMock

Abstract: Unit testing is an important part of the software development process. Good unit testing can help us find problems earlier and ensure the stable operation of the system. A single test is still a good documentation. We can often understand the author's design intention for the class by looking at the single test case.

Introduction: Code refactoring is also inseparable from unit testing. Rich single testing cases will make us confident when refactoring code. Although the single test is so important, I have not been very clear about its operation principle, and I don’t know why it is necessary to configure this or that. After all, this will not work, so I am going to spend time exploring the principle of the single test and record it here.

Foreword

Unit testing is an important part of the software development process. A good unit test can help us find problems earlier and ensure the stable operation of the system. A single test is still a good documentation. We can often understand the author's design intention for the class by looking at the single test case. Code refactoring is also inseparable from unit testing. Rich single testing cases will make us confident when refactoring code. Although the single test is so important, I have not been very clear about its operation principle, and I don’t know why it is necessary to configure this or that. After all, this will not work, so I am going to spend time exploring the principle of the single test and record it here.

What happens when running unit tests in IDEA?

First, let's see what IDEA does for us when we run the singleton directly through IDEA:

Compile the project source code and test source code and output it to the target directory

Run com.intellij.rt.junit.JUnitStarter through the java command, the version of junit and the name of the single test case are specified in the parameters
java com.intellij.rt.junit.JUnitStarter -ideVersion5 -junit4 fung.MyTest,test
Here we focus on chasing the code of JUnitStarter, which is in the junit-rt.jar plugin package provided by IDEA, the specific directory: /Applications/IntelliJ IDEA.app/Contents/plugins/junit/lib/junit-rt.jar. This package can be introduced into our own project to facilitate reading the source code:



The main function of JUnitStarter

public static void main(String[] args) {
List argList = new ArrayList(Arrays.asList(args));
ArrayList listeners = new ArrayList();
String[] name = new String[1];
String agentName = processParameters(argList, listeners, name);
if (!"com.intellij.junit5.JUnit5IdeaTestRunner".equals(agentName) && !canWorkWithJUnitVersion(System.err, agentName)) {
System.exit(-3);
}

if (!checkVersion(args, System.err)) {
System.exit(-3);
}
String[] array = (String[])argList.toArray(new String[0]);
int exitCode = prepareStreamsAndStart(array, agentName, listeners, name[0]);
System.exit(exitCode);
}
There are two main methods here

...
// Processing parameters, mainly used to determine which version of the junit framework to use, and fill in the listeners according to the input parameters
String agentName = processParameters(argList, listeners, name);
...
// start the test
int exitCode = prepareStreamsAndStart(array, agentName, listeners, name[0]);
...
Next, let's take a look at the sequence diagram of the operation of the prepareStreamsAndStart method. Here is an example of JUnit4:



When IDEA confirms the framework version to be started, it will create an instance of IdeaTestRunner through the fully qualified name of the class. Taking JUnit4 as an example here, IDEA will instantiate the com.intellij.junit4.JUnit4IdeaTestRunner class object and call its startRunnerWithArgs method. In this method, it will build org.junit.runner.Request through the buildRequest method, and obtain org.junit through the getDescription method. runner.Description, and finally create an instance of org.junit.runner.JUnitCore and call its run method.

In short, IDEA will eventually use the capabilities of the Junit4 framework to start and run single test cases, so it is necessary to do some in-depth exploration of the source code of the Junit4 framework.

Junit4 source code exploration

Junit is a unit testing framework written in the Java language and has been widely used in the industry. Its authors are the famous Kent Beck and Erich Gamma. The former is the author of "Refactoring: Improving the Design of Existing Code" and "Test Driven Development". The author, the latter is the author of "Design Patterns", the father of Eclipse. Junit4 was released in 2006. Although it is an old antique, the design concepts and ideas contained in it are not outdated, and it is necessary to seriously explore it.

First, let's start with a simple single test case:

public class MyTest {
public static void main(String[] args) {
JUnitCore runner = new JUnitCore();
Request request = Request.aClass(MyTest.class);
Result result = runner.run(request.getRunner());
System.out.println(JSON.toJSONString(result));
}
@Test
public void test1() {
System.out.println("test1");
}
@Test
public void test2() {
System.out.println("test2");
}
@Test
public void test3() {
System.out.println("test3");
}
}
Here we no longer start the unit test through IDEA's plug-in, but directly through the main function. The core code is as follows:

public static void main(String[] args) {
// 1. Create an instance of JUnitCore
JUnitCore runner = new JUnitCore();
// 2. Build Request from the Class object of the unit test class
Request request = Request.aClass(MyTest.class);
// 3. Run unit tests
Result result = runner.run(request.getRunner());
// 4. print the result
System.out.println(JSON.toJSONString(result));
}
Focus on runner.run(request.getRunner()), first look at the code of the run function:



It can be seen that which type of test process is finally run depends on the incoming runner instance, that is, different runners determine different running processes. You can guess by the name of the implementation class. JUnit4ClassRunner should be the basic test process of JUnit4. MockitoJUnitRunner should introduce the ability of Mockito, SpringJUnit4ClassRunner should have some connection with Spring and may start the Spring container.

Now, let's go back and look at the code of request.getRunner() in runner.run(request.getRunner()):

public Runner getRunner() {
if (runner == null) {
synchronized (runnerLock) {
if (runner == null) {
runner = new AllDefaultPossibilitiesBuilder(canUseSuiteMethod).safeRunnerForClass(fTestClass);
}
}
}
return runner;
}
public Runner safeRunnerForClass(Class testClass) {
try {
return runnerForClass(testClass);
} catch (Throwable e) {
return new ErrorReportingRunner(testClass, e);
}
}
public Runner runnerForClass(Class testClass) throws Throwable {
List builders = Arrays.asList(
ignoredBuilder(),
annotatedBuilder(),
suiteMethodBuilder(),
junit3Builder(),
junit4Builder()
);
for (RunnerBuilder each : builders) {
Runner runner = each.safeRunnerForClass(testClass);
if (runner != null) {
return runner;
}
}
return null;
}
You can see that the Runner is selected based on the information of the incoming test class (testClass). The rules here are as follows:

If parsing fails, return ErrorReportingRunner
Returns IgnoredClassRunner if the test class is annotated with @Ignore
If there is a @RunWith annotation on the test class, instantiate a Runner with the value of @RunWith and return
If canUseSuiteMethod=true, return SuiteMethod, which inherits from JUnit38ClassRunner, which is an earlier version of JUnit
Returns JUnit38ClassRunner if the JUnit version is before 4
If none of the above is satisfied, return BlockJUnit4ClassRunner, which represents a standard JUnit4 test model
The simple example we gave earlier returns BlockJUnit4ClassRunner, then take BlockJUnit4ClassRunner as an example to see how its run method is executed.

First, it will go to the run method in its parent class ParentRunner

@Override
public void run(final RunNotifier notifier) ​​{
EachTestNotifier testNotifier = new EachTestNotifier(notifier,
getDescription());
try {
Statem ent statement = classBlock(notifier);
statement.evaluate();
} catch (AssumptionViolatedException e) {
testNotifier.addFailedAssumption(e);
} catch (StoppedByUserException e) {
throw e;
} catch (Throwable e) {
testNotifier.addFailure(e);
}
}
It is necessary to expand the Statement here. The official explanation is: Represents one or more actions to be taken at runtime in the course of running a JUnit test suite.

Statement can be simply understood as the encapsulation and abstraction of executable methods. For example, RunBefores is a Statement, which encapsulates all methods marked with the @BeforeClass annotation. These methods are executed before running the use case of the singleton class. After running, RunBefores also returns Subsequent Statements will be run through next.evaluate(). Here are some common Statements:

RunBefores will first run the method encapsulated in befores (usually marked with @BeforeClass or @Before), and then run next.evaluate()
RunAfters, will run next.evaluate() first, and then run the method encapsulated in afters (usually marked with @AfterClass or @After)
InvokeMethod, directly run the method encapsulated in testMethod
It can be seen that the running process of the entire single test is actually the running process of a series of Statements. Taking the previous MyTest as an example, the execution process of its Statement can be roughly summarized as follows:



There is one final question left, how is the actual method under test executed? The answer is a reflective call. The core code is as follows:

@Override
public void evaluate() throws Throwable {
testMethod.invokeExplosively(target);
}
public Object invokeExplosively(final Object target, final Object... params)
throws Throwable {
return new ReflectiveCallable() {
@Override
protected Object runReflectiveCall() throws Throwable {
return method.invoke(target, params);
}
}.run();
}
So far, the execution process of a standard Junit4 single test case has been analyzed, so how does a single test like Spring that requires a container run? Let's explore it next.

Exploration of Spring Single Test
Let's start with a simple example

@RunWith(SpringRunner.class)
@ContextConfiguration(locations = { "/spring/spring-mybeans.xml" })
public class SpringRunnerTest {
@Autowired
private MyTestBean myTestBean;
@Test
public void test() {
myTestBean.test();
}
}
Here's a rough overview of what happens when you run a single test. First, @RunWith annotates the test class, so the Junit framework will first create an instance of SpringRunner with SpringRunnerTest.class as a parameter, and then call SpringRunner's run method to run the test, which will start the Spring container and load the Bean specified by the @ContextConfiguration annotation. The configuration file will also process the @Autowired annotation to inject myTestBean for the instance of SpringRunnerTest, and finally run the test() test case.

In short, start the Spring container through SpringRunner, and then run the test method. Next, explore the process of SpringRunner starting the Spring container.

public final class SpringRunner extends SpringJUnit4ClassRunner {
public SpringRunner(Class clazz) throws InitializationError {
super(clazz);
}
}
public class SpringJUnit4ClassRunner extends BlockJUnit4ClassRunner {
...
}
SpringRunner and SpringJUnit4ClassRunner are actually equivalent. It can be considered that SpringRunner is an alias of SpringJUnit4ClassRunner. Here we focus on the implementation of the SpringJUnit4ClassRunner class.

SpringJUnit4ClassRunner inherits BlockJUnit4ClassRunner. The BlockJUnit4ClassRunner has been analyzed before. It runs a standard JUnit4 test model. SpringJUnit4ClassRunner has made some extensions on this basis. The extended content mainly includes:

The constructor is extended to create one more instance of TestContextManager.
The createTest() method is extended, and the prepareTestInstance method of TestContextManager will be called additionally.
Extending beforeClass, before executing the @BeforeClass annotated method, the beforeTestClass method of TestContextManager will be called first.
Extending before, before executing the @Before annotated method, the beforeTestMethod method of TestContextManager will be called first.
After Class is extended, after executing the method annotated with @AfterClass, the afterTestClass method of TestContextManager will be called again.
After extending after, after executing the @After annotated method, the after method of TestContextManager will be called again.
TestContextManager is the core class of Spring Test Framework. The official explanation is: TestContextManager is the main entry point into the Spring TestContext Framework. Specifically, a TestContextManager is responsible for managing a single TestContext.

TestContextManager manages TestContext, and TestContext is a re-encapsulation of ApplicationContext. TestContext can be understood as a Spring container that adds test-related functions. TestContextManager also manages TestExecutionListeners, where the observer mode is used to provide the ability to monitor key nodes (such as beforeClass, afterClass, etc.) in the test running process.

So by studying the code of the relevant implementation classes of TestContextManager, TestContext and TestExecutionListeners, it is not difficult to find the startup secret of the Spring container during testing. The key code is as follows:

public class DefaultTestContext implements TestContext {
...
public ApplicationContext getApplicationContext() {
ApplicationContext context = this.cacheAwareContextLoaderDelegate.loadContext(this.mergedContextConfiguration);
if (context instanceof ConfigurableApplicationContext) {
@SuppressWarnings("resource")
ConfigurableApplicationContext cac = (ConfigurableApplicationContext) context;
Assert.state(cac.isActive(), () ->
"The ApplicationContext loaded for [" + this.mergedContextConfiguration +
"] is not active. This may be due to one of the following reasons: " +
"1) the context was closed programmatically by user code; " +
"2) the context was closed during parallel test execution either " +
"according to @DirtiesContext semantics or due to automatic eviction " +
"from the ContextCache due to a maximum cache size policy.");
}
return context;
}
...
}
In the getApplicationContext method of DefaultTestContext, the loadContext of the cacheAwareContextLoaderDelegate is called, and finally the refresh method of the Context is transferred to build the Spring container context. The timing diagram is as follows:



So where is the getApplicationContext method called?

As mentioned earlier, TestContextManager extends the createTest() method and will additionally call its prepareTestInstance method.

public void prepareTestInstance(Object testInstance) throws Exception {
if (logger.isTraceEnabled()) {
logger.trace("prepareTestInstance(): instance [" + testInstance + "]");
}
getTestContext().updateState(testInstance, null, null);
for (TestExecutionListener testExecutionListener : getTestExecutionListeners()) {
try {
testExecutionListener.prepareTestInstance(getTestContext());
}
catch (Throwable ex) {
if (logger.isErrorEnabled()) {
logger.error("Caught exception while allowing TestExecutionListener [" + testExecutionListener +
"] to prepare test instance [" + testInstance + "]", ex);
}
ReflectionUtils.rethrowException(ex);
}
}
}
The prepareTestInstance method of all TestExecutionListeners will be called in the prepareTestInstance method, and one of the listeners called DependencyInjectionTestExecutionListener will be called to the getApplicationContext method of TestContext.

public void prepareTestInstance(TestContext testContext) throws Exception {
if (logger.isDebugEnabled()) {
logger.debug("Performing dependency injection for test context [" + testContext + "].");
}
injectDependencies(testContext);
}
protected void injectDependencies(TestContext testContext) throws Exception {
Object bean = testContext.getTestInstance();
Class clazz = testContext.getTestClass();

// Call the getApplicationContext method of TestContext here to build the Spring container
AutowireCapableBeanFactory beanFactory = testContext.getApplicationContext().getAutowireCapableBeanFactory();

beanFactory.autowireBeanProperties(bean, AutowireCapableBeanFactory.AUTOWIRE_NO, false);
beanFactory.initializeBean(bean, clazz.getName() + AutowireCapableBeanFactory.ORIGINAL_INSTANCE_SUFFIX);
testContext.removeAttribute(REINJECT_DEPENDENCIES_ATTRIBUTE);
}
One last question left, how is DependencyInjectionTestExecutionListener added? The answer is spring.factories



So far, the startup process of Spring single test has been explored and understood. Next, let's take a look at SpringBoot.

Exploration of SpringBoot Single Test
A simple SpringBoot unit test example

@RunWith(SpringRunner.class)
@SpringBootTest(classes = Application.class)
public class MySpringBootTest {
@Autowired
private MyTestBean myTestBean;
@Test
public void test() {
myTestBean.test();
}
}

@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
@Documented
@Inherited
@BootstrapWith(SpringBootTestContextBootstrapper.class)
public @interface SpringBootTest {
...
}
The rough filter explains that the test is still started through SpringRunner's run method, which will start the Spring container, while @SpringBootTest provides a startup class, and at the same time, the SpringBootTestContextBootstrapper class provided by @BootstrapWith enriches the capabilities of TestContext, making it support SpringBoot's some features. Here we focus on the @BootstrapWith annotation and SpringBootTestContextBootstrapper.

When I introduced TestContextManager earlier, I didn't mention its constructor and the instantiation process of TestContext, so I will add it here.

public TestContextManager(Class testClass) {
this(BootstrapUtils.resolveTestContextBootstrapper(BootstrapUtils.createBootstrapContext(testClass)));
}
public TestContextManager(TestContextBootstrapper testContextBootstrapper) {
this.testContext = testContextBootstrapper.buildTestContext();
registerTestExecutionListeners(testContextBootstrapper.getTestExecutionListeners());
}
public abstract class AbstractTestContextBootstrapper implements TestContextBootstrapper {
...
public TestContext buildTestContext() {
return new DefaultTestContext(getBootstrapContext().getTestClass(), buildMergedContextConfiguration(),
getCacheAwareContextLoaderDelegate());
}
...
}
Building DefaultTestContext requires 3 parameters:

testClass, the class metadata being tested
MergedContextConfiguration, which encapsulates the annotations related to the test container declared on the test class, such as @ContextConfiguration, @ActiveProfiles, @TestPropertySource
CacheAwareContextLoaderDelegate, used for loading or closing containers
So what should we do when we need to extend the functionality of TestContext, or don't want to use DefaultTestContext? The easiest way is to write a new class to implement the TestContextBootstrapper interface and override the buildTestContext() method, so how to tell the test framework to use the new implementation class? That's where @BootstrapWith comes in handy. Here's a look at the code of BootstrapUtils.resolveTestContextBootstrapper

static TestContextBootstrapper resolveTestContextBootstrapper(BootstrapContext bootstrapContext) {
Class testClass = bootstrapContext.getTestClass();
Class clazz = null;
try {
clazz = resolveExplicitTestContextBootstrapper(testClass);
if (clazz == null) {
clazz = resolveDefaultTestContextBootstrapper(testClass);
}
if (logger.isDebugEnabled()) {
logger.debug(String.format("Instantiating TestContextBootstrapper for test class [%s] from class [%s]",
testClass.getName(), clazz.getName()));
}
TestContextBootstrapper testContextBootstrapper =
BeanUtils.instantiateClass(clazz, TestContextBootstrapper.class);
testContextBootstrapper.setBootstrapContext(bootstrapContext);
return testContextBootstrapper;
}
...
}
private static Class resolveExplicitTestContextBootstrapper(Class testClass) {
Set annotations = AnnotatedElementUtils.findAllMergedAnnotations(testClass, BootstrapWith.class);
if (annotations.isEmpty()) {
return null;
}
if (annotations.size() == 1) {
return annotations.iterator().next().value();
}
// Get the value of the @BootstrapWith annotation
BootstrapWith bootstrapWith = testClass.getDeclaredAnnotation(BootstrapWith.class);
if (bootstrapWith != null) {
return bootstrapWith.value();
}
throw new IllegalStateException(String.format(
"Configuration error: found multiple declarations of @BootstrapWith for test class [%s]: %s",
testClass.getName(), annotations));
}
Here, the customized TestContextBootstrapper will be instantiated through the value of the @BootstrapWith annotation to provide a customized TestContext

SpringBootTestContextBootstrapper is the implementation class of TestContextBootstrapper, which indirectly inherits AbstractTestContext The Bootstrapper class extends the ability to create TestContext, these extensions mainly include:

Replaced ContextLoader with SpringBootContextLoader
Added DefaultTestExecutionListenersPostProcessor for enhanced processing of TestExecutionListener
Added handling of webApplicationType
Next, look at the relevant code of SpringBootContextLoader

public class SpringBootContextLoader extends AbstractContextLoader {
@Override
public ApplicationContext loadContext(MergedContextConfiguration config)
throws Exception {
Class[] configClasses = config.getClasses();
String[] configLocations = config.getLocations();
Assert.state(
!ObjectUtils.isEmpty(configClasses)
|| !ObjectUtils.isEmpty(configLocations),
() -> "No configuration classes"
+ "or locations found in @SpringApplicationConfiguration. "
+ "For default configuration detection to work you need"
+ "Spring 4.0.3 or better (found " + SpringVersion.getVersion()
+ ").");
SpringApplication application = getSpringApplication();
// set mainApplicationClass
application.setMainApplicationClass(config.getTestClass());
// set primarySources
application.addPrimarySources(Arrays.asList(configClasses));
// add configLocations
application.getSources().addAll(Arrays.asList(configLocations));
// get environment
ConfigurableEnvironment environment = getEnvironment();
if (!ObjectUtils.isEmpty(config.getActiveProfiles())) {
setActiveProfiles(environment, config.getActiveProfiles());
}
ResourceLoader resourceLoader = (application.getResourceLoader() != null)
? application.getResourceLoader()
: new DefaultResourceLoader(getClass().getClassLoader());
TestPropertySourceUtils.addPropertiesFilesToEnvironment(environment,
resourceLoader, config.getPropertySourceLocations());
TestPropertySourceUtils.addInlinedPropertiesToEnvironment(environment,
getInlinedProperties(config));
application.setEnvironment(environment);
// Get and set initializers
List> initializers = getInitializers(config,
application);
if (config instanceof WebMergedContextConfiguration) {
application.setWebApplicationType(WebApplicationType.SERVLET);
if (!isEmbeddedWebEnvironment(config)) {
new WebConfigurer().configure(config, application, initializers);
}
}
else if (config instanceof ReactiveWebMergedContextConfiguration) {
application.setWebApplicationType(WebApplicationType.REACTIVE);
if (!isEmbeddedWebEnvironment(config)) {
new ReactiveWebConfigurer().configure(application);
}
}
else {
application.setWebApplicationType(WebApplicationType.NONE);
}
application.setInitializers(initializers);
// Run the SpringBoot application
return application.run();
}
}
You can see that SpringApplication is built here, mainApplicationClass is set, primarySources is set, initializers are set, and the SpringBoot application is finally started through application.run().



So far, the startup process of the SpringBoot unit test has also been explored and understood. Next, let's see how the Maven plugin runs the unit test.

How the Maven plugin runs a single test
We know that maven helps us complete the construction, testing, packaging, deployment and other actions in the project development process through a series of plug-ins. When running the maven clean test command in the Console, maven will run the following goals in sequence:

maven-clean-plugin:2.5:clean, used to clean the target directory
maven-resources-plugin:2.6:resources, move the resource files in the main project directory to the classes directory in the target directory
maven-compiler-plugin:3.1:compile, compile the java source code in the main project directory into bytecode, and move it to the classes directory in the target directory
maven-resources-plugin:2.6:testResources, move the resource files in the test project directory to the test-classes directory in the target directory
maven-compiler-plugin:3.1:testCompile, compile the java source code in the test project directory into bytecode, and move it to the classes directory in the target directory
maven-surefire-plugin:2.12.4:test, run single test
Let's take a look at the code of the maven-surefire-plugin plugin. First, introduce the maven-surefire-plugin and surefire-junit4 packages so that we can view the code:


org.apache.maven.plugins
maven-surefire-plugin
2.9


org.apache.maven.surefire
surefire-junit4
3.0.0-M7

The core code is in org.apache.maven.plugin.surefire.AbstractSurefireMojo#execute. The code will not be posted here. If you are interested, you can look at it yourself. In short, the information in JUnit4ProviderInfo will be used to instantiate the JUnit4Provider object through reflection, and then call its invoke method. In the modified method, the Runner will finally be instantiated and its run method will be called. The core code is as follows:

private static void execute( Class testClass, Notifier notifier, Filter filter )
{
final int classModifiers = testClass.getModifiers();
if ( !isAbstract( classModifiers ) && !isInterface( classModifiers ) )
{
Request request = aClass( testClass );
if ( filter != null )
{
request = request.filterWith( filter );
}
Runner runner = request.getRunner();
if ( countTestsInRunner( runner.getDescription() ) != 0 )
{
runner.run(notifier);
}
}
}

Summarize

So far, the relevant principles of unit test operation have been explored. Let's review what is included.

When running a unit test directly through IDEA, the main method of JUnitStarter is used as the entry point, and Junit is finally called to run the unit test.
Junit4 abstracts the annotation marking methods of @Before, @Test, and @After into Statements. The entire running process of a single test is actually the running process of a series of Statements. Method invocation is achieved through reflection.
With the @RunWith(SpringRunner.class) annotation, the test framework will run the run method of the SpringRunner instance, create the TestContext through the TestContextManager, and start the Spring container. SpringRunner and SpringJUnit4ClassRunner are actually equivalent.
With the help of @SpringBootTest and @BootstrapWith(SpringBootTestContextBootstrapper.class) annotations, the test framework passes SpringBootTestContextBootstrapper Enhanced TestContext to achieve the purpose of starting SpringBoot applications.
Maven starts the unit test by running maven-surefire-plugin:2.12.4:test, the core of which is the code that calls the JUnit framework through the JUnit4Provider.