The Most Lightweight New Mock Tool of Alibaba for Unit Testing Is Open-Source!

By Jinji

Introduction

The Alibaba Cloud Yunxiao Team tried to make the mock definition and replacement easier based on the mainstream mock tools to explore more lightweight and easy-to-use mock test methods. A simple test tool, TestableMock, was released. It does not require initialization or testing framework. It can replace all kinds of methods, including private methods, static methods, or construction methods, and does not concern the construction of objects to be replaced. The replacement can be completed by writing the mock definition and adding a @MockMethod annotation. This article introduces the implementation principle of TestableMock.

How should the simplest and most comfortable mock test work?

It should be like pointing at the code where the source file calls external dependency and saying, "Replace it with this fake call when testing!"

Done.

Then, replace it directly without any unnecessary actions.

The Eight Steps of Mock Test

Since Java mock tools have been continuously iterated and developed with the unit testing technology, their principles vary, but the core usage mode has hardly changed. The basic usage mode is the same for (the currently popular) Mockito and PowerMock or (the once famous) JMockit, EasyMock, and MockRunner. First, they conduct initialization. Then, they define mock objects. Next, they send back the defined mock object to the class under test through a mechanism to replace the originally called object.

The following is the code from the Mockito test.

// Step 1: initialize Mockito
@RunWith(MockitoJUnitRunner.class)
public class RecordServiceTest 
{     
    // Step 2: define a mock object
    @Mock
    DatabaseDAO databaseMock;

    // Step 3: define test cases
    @Test
    public void saveTest()
    {
        // Step 4: define alternative methods
        when(databaseMock.write()).thenReturn(4);
        // Step 5: inject mock object
        RecordService recordService =
            new RecordService(databaseMock);

        // Step 6: execute test content
        boolean saved = recordService.save("demo");

        // Step 7: verify test results
        assertEquals(true, saved);
        // Step 8: verify whether the mock method is executed
        verify(databaseMock, times(1)).write();
    }
}

Based on different implementation principles, there are many ways to send the mock object back to the method under test.

Based on Dynamic Proxy, Mockito is more intuitive, but it does not help a lot. Spring Bean supports @Autowired with @InjectMocks. Therefore, the user code must be "testable." If the object to be replaced does not use dependency injection, Mockito doesn't work.

PowerMock is based on the customized loader and can replace the mock object through @PrepareForTest. However, the default test coverage rate of the on-the-fly mode of Jacoco will drop to zero. The usage procedure of PowerMock is very similar to Mockito but with more functions and a steeper learning curve for developers.

Based on dynamic bytecode modification, JMockit is better. It allows the replacement of mock objects without influencing the test coverage. However, JMockit requires each case to use a fixed structure at the beginning and end. It has also invented a mock definition syntax that is not in line with Java's habits. For example:

// Step 1: initialize JMockit
@RunWith(JMockit.class)
public class PerformerTest {
 
    // Step 2: define a mock object
    @Mocked
    private Collaborator collaborator;
 
    // Step 3: define the object to be tested
    // Implicitly inject the logic of the mock object
    @Tested
    private Performer performer;
 
    // Step 4: define test cases
    @Test
    public void testThePerformMethod() {
        // Step 5: define alternative methods
        new Expectations() {{
            collaborator.work("bar"); result = 10;
        }};

        // Step 6: execute test content
        boolean res = performer.perform("test");
        // Step 7: verify test results
        assertEquals(true, res);

        // Step 8: verify whether the mock method is executed
        new Verifications() {{
            collaborator.receive(true);
        }};
    }
}

The usage procedures of other mock tools are almost the same and will not be introduced here. This magical rule shows that any complete mock test process follows a fixed structure where five of the eight steps are related to Mock.

Mock tools were only expected to come into play in external dependence. How did it control the entire test structure?

Extremely Simple TestableMock

We tried to reduce the burden on the tool to explore a more lightweight and easy-to-use mock test method. We want to make the mock definition and replacement easier, so we designed a simple test auxiliary tool, TestableMock. For the open-source TestableMock, please see: https://github.com/alibaba/testable-mock

In TestableMock's world, mock specifies the target method, defines the alternative method, and watches the target method get automatically replaced while testing. Only one annotation, @MockMethod, is required. If the first example above is implemented via TestableMock, it could look like this:

public class RecordServiceTest 
{
    // Define mock target and alternative methods inside an inner class
    public static class Mock {
        // The mock method has one more parameter than the original method and is passed into the caller.
        // So it replaces the int write() method call of the DatabaseDAO class
        @MockMethod
        int write(DatabaseDAO origin) { return 4; }
    }

    // Define test cases
    @Test
    public void saveTest()
    {
        // Execute test content
        RecordService rs = new RecordService();
        boolean saved = rs.save("demo");

        // Verify test results
        assertEquals(true, saved);
        // Verify whether the mock method is executed
        TestableTool.verify("write").times(1);
    }
}

There are five steps, but only two are related to Mock. There is no need to initialize the framework, intrude into test cases for mock definitions, or worry about how mock methods are injected. The @MockMethod annotation manages everything. In the tested class, all calls of the DatabaseDAO object write() method are replaced with empty calls and return the value "4".

Unlike the previous mock tools that always replace the entire object, TestableMock replaces the target method directly. This simplified design is mainly based on two assumptions:

Assumption 1: In the same tested class, if methods in one test case are replaced by Mock, these methods will be replaced with other cases. These methods often access external dependencies that are not easily tested.
Assumption 2: All calls that need to be replaced by mock are codes of the tested class. This assumption is in line with the idea of unit testing. Unit testing should only focus on the internal behavior of the current unit, while the logic outside the unit should be replaced by Mock.

For assumption 1, TestableMock allows a small number of special cases. For example, in the above mock method replacement, TestableTool can be used to assist the judgment only if the write() call in the save method will be replaced.

@MockMethod
int write(DatabaseDAO origin) {
    switch(TestableTool.SOURCE_METHOD) {
        case "save": return 10;
        default: return origin.write();
    }
}

Normally, assumption 2 should not have any special cases. Otherwise, there is something wrong with the unit test code.

The "lightweight" feature of TestableMock has no appointed partners. The code does not customize the logic for any running framework or test framework. It comes into play whether the project uses Spring, JFinal, Quarkus, JUnit4, JUnit5, TestNG, Jacoco, or other tools. In addition, TestableMock can replace private methods, static methods, and new operators of the tested class, except for method calls of objects. The new replaced operator can return either a real object or a mock object encapsulated by Dynamic Proxy. However, TestableMock is not responsible for generating such mock objects because traditional mock tools, such as Mockito, are responsible for this.

TestableMock, as a mock tool, can simplify all of the preparations required for mock replacement. Does it have any shortcomings compared with the traditional mock tools? TestableMock does not introduce any major bottom-layer new technologies. It follows an unwritten law that any non-disruptive improvement is a trade-off. Though being extremely simple, TestableMock is not applicable for the two scenarios in the two assumptions above. The mock method and test cases are defined separately. Therefore, if there are too many "if" and "switch" in the mock method and it needs to distinguish the call source, the code logic is unclear. However, such cases are uncommon. It is more common that many test cases need to use the same mock method. If mock definition is separated in this case, it will be more helpful to reduce duplicate code, which does more good than harm.

The Principle of TestableMock

In short, TestableMock uses the runtime bytecode modification technology. It scans the bytecode of the test class and the tested class during unit testing startup to complete the mock method replacement.

This technology selection considers TestableMock's pursuit of complete functions and lightweight design.

In real cases, the principles of mock tools for Java unit testing can be divided into three types, and the respective typical tools are listed below:

Dynamic Proxy: Mockito, EasyMock, and MockRunner
Customized Class Loader: PowerMock
Runtime Bytecode Modification: JMockit and TestableMock

Among the three types of principles, Dynamic Proxy only changes the periphery of the tested class. It is the safest principle, but the function is less powerful. These mock tools are picky about mock methods, while final types, static methods, and private methods cannot be covered.

Both Customized Class Loader and Runtime Bytecode Modification modify the bytecode of the tested class. The former completely takes over the loading process, while the latter performs "secondary modification" of the bytecode after class loading. These two are not much different in function and can implement mock replacement of almost all types and methods. The main difference lies in the enabling way. Special processing is needed for different testing frameworks to let the Customized Class Loader take effect. For example, the @RunWith annotation is needed in JUnit. In PowerMock, the annotation needed for different frameworks varies.

TestableMock automatically determines the demand for corresponding initialization to be completely decoupled from the testing framework. To do so, it directly scans if the test class contains methods modified by @MockMethod (or @MockConstructor). This achieves mock initialization, definition, and replacement with only one annotation. Mock replacement is performed with reusable methods rather than the entire type. Thus, the whole process is free from test code invasion.

Are there other mock implementation methods? TestableMock in an earlier version also tried Pluggable Annotation Processing of JSR-269 specification to modify the compiled source code during code compilation. This mechanism can also replace the method call in the source code with the mock call, but it brings two problems. Firstly, the modified source code will be packaged into the final jar package, causing tampering issues with the production package content. This can be solved by restoring the class file before packaging, but it is inefficient. Secondly, if the source code is modified, the replacement based on each JVM language must be implemented separately. During the iteration, TestableMock gradually abandons the mock solution based on JSR-269 and uses Pluggable Annotation Processing to implement another function, access for private members of the tested class.

Be More Than a mock Tool

TestableMock, developed by the Alibaba Cloud Yunxiao Team, upholds the philosophy to make R&D simpler. It aims to "make all Java methods easily testable", which also reflects the meaning of the name.

In addition to unique mock functions, TestableMock provides three unit testing enhancements:

Allows the test case of unit testing to access private members of the tested class directly

Testing private methods have always been controversial in Java unit testing. In the Java ecosystem, new programming languages, such as Python, Golang, and Rust, avoid this argument from the beginning. Python's "private method" is just a naming convention. In Golang, all methods in the same package are accessible by default. The unit testing of Rust is accompanied by the code under test. These new languages have enabled unit testing to access private methods by default. For Java code, it has to change the visibility of private methods to default or public for testing, which destroys the encapsulation and triggers the argument. However, the way of "indirectly testing private methods through public methods" will make it hard for testers to operate in practice. TestableMock provides a @EnablePrivateAccess annotation to enhance the accessibility. TestableMock allows all private member codes that access the corresponding tested class to be automatically replaced with valid reflection calls during compilation. Access to private methods in other classes is still not allowed.

Support easyly construct complicated parameter objects

In unit testing, the preparation and construction of test data is a necessary and tedious task. Object-oriented layer-by-layer encapsulation becomes an obstacle to initializing the state of the object during testing. Especially when the type structure is complicated, there is no suitable construction method, or some fields need to use private inner classes, etc. Using conventional methods to construct those class often appear to be inadequate. For this reason, TestableMock provides two minimalist tool classes, OmniConstructor and OmniAccessor, which makes the construction of any object no longer difficult.

Assist the test of void type methods without returned values

"Testing methods without returned values" is a technical issue with little disagreement, but there are no simple and practical solutions so far. Although the void type methods do not directly return the computing results, they will cause a global state change or a "function side effect," such as log output and external systems call. Methods that do not return data or produce any side effects are worthless. The access mechanism for private members and the mock validator of TestableMock enable quick verification of internal state changes of the tested class. They also verify the execution and parameter input of call statements that have side effects in tested methods. Therefore, void type methods of Java projects may have easier testing.

Summary

TestableMock is not inferior to PowerMock in functions and simpler than Mockito in usage. It does its job with only a @MockMethod annotation.

Unit testing is an effective method to ensure code refactoring and avoid code degradation. However, in practice, many developers lose their confidence because of the rules and regulations of unit testing and the cost of code writing. The pragmatic and enhanced unit testing tool, TestableMock provides the powerful mock replacement capability and reduces all code writing costs to a record low.

Bring back the original nature of mock and leave out the cumbersome tests. For the open-source TestableMock, please see: https://github.com/alibaba/testable-mock/blob/master/README_EN.md

Community

The Most Lightweight New Mock Tool of Alibaba for Unit Testing Is Open-Source!

Introduction

The Eight Steps of Mock Test

Extremely Simple TestableMock

The Principle of TestableMock

Be More Than a mock Tool

Summary

Read previous post:

Read next post:

Alibaba Clouder

You may also like

Comments

Alibaba Clouder

Related Products

Mobile Testing

Storage Capacity Unit

Penetration Test

Super App Solution for Telcos