By Chengpu and Cengfeng
With the rapid growth of cloud-native technology, providing the ultimate flexibility for enterprise applications has become a core requirement for enterprise digital upgrades. However, as a language with interpretation, execution, and runtime Just-In-Time (JIT) compilation, Java has inherent disadvantages compared to other statically compiled languages, significantly impacting its quick startup and scaling efficiency.
The detailed process of starting and running a Java program is shown in Figure 1:
Figure 1 Startup process analysis of Java programs[1]
When a Java application starts up, the JVM must first be loaded into memory, as depicted in the red section of Figure 1 above. Subsequently, the JVM loads the corresponding application into memory, represented by the light blue class loading (CL) section in the figure. During the class loading process, the application begins to be interpreted and executed, as indicated by the light green part in the figure. Within the interpretation and execution process, the JVM performs garbage collection on objects, as shown in the yellow section of the figure.
As the program progresses, the JVM employs JIT compilation technology to compile and optimize frequently executed code, enhancing the application's running speed. The JIT process is illustrated by the white portion in the figure, and the code optimized by JIT compilation is denoted by the dark green part. Thus, it is evident from this analysis that a Java program goes through several stages, including VM initialization, App initialization, and App activation before JIT dynamic compilation optimization is reached. In comparison to other compiled languages, Java's cold start issue is more severe.
Apart from the cold start issue, Figure 1 indicates that during the execution of a Java program, the initial step involves loading a certain amount of memory with a JVM. Additionally, JIT compilation and garbage collection will have a certain amount of memory overhead.
Lastly, since Java programs first interpret and execute bytecode before performing JIT compilation optimization, some unnecessary code logic might also be pre-loaded into memory for compilation due to its late compilation. Therefore, in addition to the actual application to be executed, the inclusion of unnecessary code logic represents an additional overhead that cannot be overlooked. These factors contribute to the widespread criticism of the high memory usage of Java programs.
The severe cold start time and high runtime memory usage make it challenging for Java applications to meet the demands of cloud-native fast startup and scaling. Consequently, the GraalVM open-source community led by Oracle Corporation introduced Java static compilation technology. This technology enables Java programs to be compiled into local executable files in advance, allowing them to achieve peak performance. It effectively addresses the issues of Java application cold start and high runtime memory usage, ensuring Java can thrive in the wave of cloud-native technology.
If you do not know static compilation technology before, you can read the article From Local-native to Cloud-native: Practices and Challenges of Alibaba Dragonwell Static Compilation[3] and Building Microservices Applications Based on Static Compilation for more detailed understanding.
While static compilation technology offers many benefits, it may have a certain impact on the existing Java technology system. For instance, those familiar with static compilation may know that Java, after static compilation, lacks bytecode, rendering some Java Agent non-intrusive bytecode rewriting technologies based on Java bytecode ineffective. This includes numerous solutions in the Java ecosystem that provide non-intrusive distributed Tracing Analysis capabilities for Java applications based on bytecode rewriting. Under existing Java static compilation solutions, these approaches will also fail, presenting technical challenges that many enterprises need to consider before implementing static compilation technologies.
Is it impossible to achieve out-of-the-box observability based on Java Agent like traditional Java applications in static compilation scenarios?
Recently, Alibaba Cloud team has implemented a static Agent instrumentation enhancement for GraalVM. They verified the correctness and integrity of static enhancement data on the Alibaba Cloud Application Real-Time Monitoring Service (ARMS) observability platform, effectively addressing the current issue of Java Agent bytecode enhancement during Java static compilation. This meets the requirements of both improving Java applications' performance based on GraalVM static compilation and implementing non-intrusive distributed Tracing Analysis and other observable effects through technologies such as Java Agent, similar to non-static compilation scenarios.
In order to understand what static instrumentation is, it is essential to discuss its relative concept: dynamic instrumentation.
For those familiar with Java Agent technology, the function process of Java Agent involves bytecode rewriting technology. During the class loading phase of the application running process, bytecode rewriting technology is used to instrument some enhanced logic before and after specific class methods (also called instrumentation) of the application. This is aimed at adding observable capabilities, such as distributed Tracing Analysis, to the application without the awareness of the application's R&D personnel.
Compared to dynamically instrumenting logic through bytecode rewriting during the execution of dynamic instrumentation applications, static instrumentation involves bytecode rewriting before the program commences. Subsequently, in the GraalVM static compilation phase before running, the final content of the collected bytecode rewriting is compiled into the final executable file. This achieves the same non-intrusive effect as dynamic instrumentation to enhance the application's capabilities at specific instrumentations.
The concept of static instrumentation indicates that to enhance the application code, it is necessary to address the following two questions:
Therefore, we have designed a pre-execution recording + compile-time replacing method to solve this problem. The overall process is divided into two steps:
This theoretically resolves the issue of where and what to enhance in the application.
Firstly, let's review the detailed functioning of the Java Agent mechanism. This consists of registering the class converter (Transformer) defined in the Agent and the hook to response eventHandlerClassFileHook into the JVM before the main function of the application is started. Every time an application first loads a class, the code registered in the eventHandlerClassFileHook is executed to load the class. Developers can modify specified classes in this hook, ensuring the loaded class during runtime is the enhanced class by Agent.
This means that for any class C, the JVM Agent mechanism guarantees that C is replaced with C' when first loaded. From the application's perspective, only C' is seen throughout runtime, not C. Therefore, assuming that we replace C with C' at compile-time, it is always C' for the application. Thus, the issue of runtime is translated into two compile-time problems:
Readers who have learned about GraalVM static compilation technology may know that GraalVM provides an agent called native-image-agent [4] which can record dynamic behaviors such as reflection, dynamic class loading, dynamic proxy, and serialization in Java applications by mounting applications for pre-execution, and output configuration files that record the information. During the compilation phase, the configuration file, as an input to the compilation, also provides the compiler with dynamic behavior information, so that the Java dynamic features can still take effect in a static compilation environment.
Therefore, by rewriting the native-image-agent, we add the logic observation and recording for Agent to implement class transformation code enhancement behavior on the original basis. The implementation principle is shown in Figure 2. The yellow Agent in the figure implements runtime dynamic enhancement on the red code C on the original application App, and converts the C part code into C', thus obtaining App'. We add the native-image-agent to record code enhancement ability. It is responsible for observing the process from C to C', saving the specific class name of C to the configuration file, and saving the transformed C' to the disk.
Figure 2 Original Agent code transformation process monitored by native-image-agent
The implementation of enhancement records through native-image-agent is the core of this scheme. The entire process must include the class name to be transformed, the byte array of the original class file, and the byte array of the transformed class file, by which we can determine whether the class has changed, so as not to record a large amount of noise information.
By organizing the workflow of JVM Agent, we have chosen the Java function sun/instrument/InstrumentationImpl.transform as the observation entry point, that is, the red circle in Figure 3. This function is hereinafter referred to as the transform function.
Figure 3 Agent implementation of dynamic transformation process supported by JVM
We add a function breakpoint for the transform function to the native-image-agent and then compare whether the class data before and after the transformation is consistent. If it is consistent, it means that no transformation has been made and the class does not need to be recorded. If it is inconsistent, it means that the class has been changed, then the fully qualified name of its class is output to the configuration file and the contents of the class are saved to the disk.
The enhanced classes are obtained, and then as long as they are used to replace the original classes at compile-time, we can achieve the effect of instrumentation enhancement in the final native image executable file which has been performed static compilation. So how do we replace it at compile-time? The simplest and safest way is to replace it during the class loading. The static compilation capability of GraalVM is also a Java program. You need to load all target compiled classes to the classpath.
So simply, we just need to put the path of the enhanced class at the front when generating the classpath list. For the case of using the module system, since the same class cannot appear in two modules, we need to prepare the enhanced class as a jar package and replace the original class in the form of --patch-module. The principle of this process is simple, but the process of automatic implementation is complex. The GraalVM static compilation framework needs to be modified, so we will not expand it here.
After processing as the above method, only the transformed code is contained in the local executable program after the GraalVM static compilation, and its runtime behavior is consistent with the expected behavior. The preceding "pre-execution recording + compile-time replacing" steps are used to implement static instrumentation in the GraalVM environment.
Based on the preceding scheme, we have verified the effect of some commonly used microservice components, such as Spring Boot, Kafka, MySQL, and Redis. Currently, we implement opentelemetry-java-instrumentation [5] for data collection directly based on the well-known observable Java Agent (hereinafter referred to as OT agent), and then report the collected observable data to the Managed Service for OpenTelemetry [6] of Alibaba Cloud Application Real-Time Monitoring Service (ARMS) for effect verification. The following is the relevant test effect.
In a general JVM runtime environment, use the OT agent to non-intrusively collect observable data from the Spring Boot application, and then report the data to the Managed Service for OpenTelemetry of Alibaba Cloud ARMS. The effect is shown in Figure 5.
Figure 5 Observable data collection and display effects under the condition of traditional JVM
During the test, we made five calls to the application. The number of calls recorded from the effect trace in Figure 5 and the trace details are consistent with those of the instance application.
In the GraalVM static compilation environment, based on the preceding scheme, use the OT agent to non-intrusively collect observable data from the Spring Boot application, and report the data to Managed Service for OpenTelemetry of Alibaba Cloud ARMS. The effect is shown in Figure 6.
Figure 6 Observable data collection and display effects under the condition of GraalVM static compilation
During the test, the application was also called five times. Through the above comparison and screenshots, it can be found that after the Spring Boot application is statically compiled based on GraalVM, the number of requests collected and other indicators are consistent with the dynamic enhancement method of the JVM environment. Thanks to the optimization of static compilation technology, the request span time (without network conditions involved) is much lower than that of the JVM environment enhancement method. In addition to the above test results of Spring Boot applications, some other common components, such as Kafka, MySQL, and Redis, have done the same test, and it was found that the scheme is effective!
In addition, the following table shows the time consumption and runtime memory usage data of some tested framework applications based on a normal JVM environment versus static compilation scenario (test environment: 32 cores (vCPU)/64 GiB/5 Mbps):
After static compilation, the startup time of each type of application is reduced by around 98%, and the runtime memory usage is reduced by approximately 70%. Based on the test results, the four framework components mentioned can benefit from the significant performance improvement brought by static compilation and also resolve the non-intrusive enhancement failure issue caused by Java Agent after static compilation.
As mentioned in the previous section, we have successfully verified the scheme and submitted the necessary modified PR [7] to the GraalVM community. If the scheme is to be implemented in a production environment, several additional engineering issues need to be addressed and optimized. For instance, there may be scenarios where the Java Agent needs to replace classes in the JDK, and GraalVM has also made modifications to some JDK classes to adapt them to runtime static compilation. Therefore, compatibility should be taken into consideration when modifications from both sides are necessary.
[1] Startup Process Analysis of Java Programs
https://shipilev.net/talks/j1-Oct2011-21682-benchmarking.pdf
[2] GraalVM Open-source Community
https://www.graalvm.org/
[3] From Local-native to Cloud-native: Practices and Challenges of Alibaba Dragonwell Static Compilation
https://www.infoq.cn/article/uzHpEbpMwiYd85jYslka
[4] native-image-agent
https://www.graalvm.org/latest/reference-manual/native-image/metadata/AutomaticMetadataCollection/
[5] opentelemetry-java-instrumentation
https://github.com/open-telemetry/opentelemetry-java-instrumentation
[6] Managed Service for OpenTelemetry
https://www.alibabacloud.com/help/en/arms/tracing-analysis/
[7] Related PR to Support Static Instrumentation
https://github.com/oracle/graal/pull/8077
Application Monitoring eBPF Edition: Non-intrusive Application Monitoring of the Golang Microservice
Better Performance and Cost-effectiveness: Migrating the Self-built ELK to SLS
484 posts | 48 followers
FollowAlibaba Cloud Community - May 10, 2024
Alibaba Cloud Native Community - December 8, 2023
Alibaba Cloud Native Community - September 13, 2023
OpenAnolis - September 28, 2022
Alibaba Clouder - May 29, 2020
Alibaba Clouder - April 15, 2021
484 posts | 48 followers
Follow
Application Real-Time Monitoring Service
Build business monitoring capabilities with real time response based on frontend monitoring, application monitoring, and custom business monitoring capabilities
Learn More
Managed Service for Prometheus
Multi-source metrics are aggregated to monitor the status of your business and services in real time.
Learn More
Content Moderation
Image and video moderation service that accurately detects inappropriate content.
Learn More
Real-Time Livestreaming Solutions
Stream sports and events on the Internet smoothly to worldwide audiences concurrently
Learn MoreMore Posts by Alibaba Cloud Native Community
