All Products
Search

This Product

    Serverless App Engine:Best practices for JVM heap size configuration

    Document Center

    Serverless App Engine:Best practices for JVM heap size configuration

    Last Updated:Jan 02, 2024

    If you configure a large Java virtual machine (JVM) heap size, the out of memory (OOM) Killer mechanism of Linux may be triggered. In this case, the Java process is terminated. In a container environment, your application may be frequently restarted. This topic describes how to configure JVM options in a container environment. This topic also provides answers to some frequently asked questions (FAQ) about OOM errors.

    (Recommended) Use the -XX:MaxRAMPercentage option to manage the heap size

    • In a container environment, the server configurations can be obtained by using Java. However, the limit on container memory is unknown. You can use the -Xmx option to manage the JVM heap size. Take note of the following items:

      • You must reconfigure the -Xmx option after you adjust the memory size.

      • If you set this option to an invalid value and the upper limit on the heap size is not reached, the container may be forcibly shut down due to OOM errors.

        Note

        If an OOM error occurs, the Linux kernel triggers the OOM Killer mechanism to release memory. The OOM Killer mechanism monitors memory-consuming processes and kills processes that consume too much memory in a short period of time to release memory for the system and prevent system crash.

    • We recommend that you configure the following JVM options: 

      -XX:+UseContainerSupport -XX:InitialRAMPercentage=70.0 -XX:MaxRAMPercentage=70.0 -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/home/admin/nas/gc-${POD_IP}-$(date '+%s').log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/admin/nas/dump-${POD_IP}-$(date '+%s').hprof

      The following table describes the JVM options.

      Option

      Description

      -XX:+UseContainerSupport

      The option to use the container memory. This option allows the JVM to read cgroup limits from the host machine, such as available CPUs and RAM, and configure itself accordingly. If the upper limit on the container memory is reached, OOM errors occur, but the container is not shut down.

      -XX:InitialRAMPercentage

      The option to specify the initial percentage of container memory that can be used by the JVM. We recommend that you set the -XX:InitialRAMPercentage option and the -XX:MaxRAMPercentage option to the same value. The recommended value is 70.0.

      -XX:MaxRAMPercentage

      The option to specify the maximum percentage of container memory that can be used by the JVM. Due to system component overheads, we recommend that you set this option to a value that is less than or equal to 75.0. The recommended value is 70.0.

      -XX:+PrintGCDetails

      The option to print GC details.

      -XX:+PrintGCDateStamps

      The option to print GC timestamps. Sample timestamp: 2019-12-24T21:53:59.234+0800

      -Xloggc:/home/admin/nas/gc-${POD_IP}-$(date '+%s').log

      The option to specify the output path for GC logs. Before you configure this option, make sure that you have a container directory to store logs. We recommend that you mount the container directory to an NAS directory or use Simple Log Service to store logs. This way, directories can be automatically created, and persistent storage is enabled for logs.

      -XX:+HeapDumpOnOutOfMemoryError

      The option to enable automatic generation of dump files when an OOM error occurs in the JVM.

      -XX:HeapDumpPath=/home/admin/nas/dump-${POD_IP}-$(date '+%s').hprof

      The option to specify the path in which dump files are stored. Before you configure this option, make sure that you have a container directory for dump files. We recommend that you mount the container directory to an NAS directory. This way, directories can be automatically created, and persistent storage is enabled for logs.

      Note

      • The -XX:+UseContainerSupport option takes effect only if you use JDK 8u191+ and JDK 10 or later.

      • If you use JDK 11, the following log options are no longer used: -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, and -Xloggc:$LOG_PATH/gc.log. We recommend that you use the -Xlog:gc:$LOG_PATH/gc.log option.

      • Dragonwell 11 does not support the ${POD_IP} variable.

      • If you do not mount the /home/admin/nas container directory to an NAS directory, make sure that the container directory exists before the application startup. Otherwise, no logs are generated.

    Use the -Xms and -Xmx options to manage the heap size

    • You can use the -Xms and -Xmx options to manage the heap size. Take note of the following items:

      • You must reconfigure the -Xmx option after you adjust the memory size.

      • If you set this option to an invalid value and the upper limit on the heap size is not reached, the container may be forcibly shut down due to OOM errors.

        Note

        If an OOM error occurs, the Linux kernel triggers the OOM Killer mechanism to release memory. The OOM Killer mechanism monitors memory-consuming processes and kills processes that consume too much memory in a short period of time to release memory for the system and prevent system crash.

    • We recommend that you configure the following JVM options: 

      -Xms2048m -Xmx2048m -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/home/admin/nas/gc-${POD_IP}-$(date '+%s').log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/admin/nas/dump-${POD_IP}-$(date '+%s').hprof

      The following table describes the JVM options.

      Option

      Description

      -Xms

      The option to specify the minimum heap size. To prevent reallocation of heap size after each garbage collection, we recommend that you set the -Xms and -Xmx options to the same value.

      -Xmx

      The option to specify the maximum heap size. To prevent OOM errors in containers, we recommend that you reserve a memory size that is sufficient to run the system.

      -XX:+PrintGCDetails

      The option to print GC details.

      -XX:+PrintGCDateStamps

      The option to print GC timestamps. Sample timestamp: 2019-12-24T21:53:59.234+0800

      -Xloggc:/home/admin/nas/gc-${POD_IP}-$(date '+%s').log

      The option to specify the output path for GC logs. Before you configure this option, make sure that you have a container directory to store logs. We recommend that you mount the container directory to an NAS directory or use Simple Log Service to store logs. This way, directories can be automatically created, and persistent storage is enabled for logs.

      -XX:+HeapDumpOnOutOfMemoryError

      The option to enable automatic generation of dump files when an OOM error occurs in the JVM.

      -XX:HeapDumpPath=/home/admin/nas/dump-${POD_IP}-$(date '+%s').hprof

      The option to specify the path in which dump files are stored. Before you configure this option, make sure that you have a container directory for dump files. We recommend that you mount the container directory to an NAS directory. This way, directories can be automatically created, and persistent storage is enabled for logs.

    • The following table describes some recommended heap size settings.

      Memory size

      JVM heap size

      1 GB

      600 MB

      2 GB

      1434 MB

      4 GB

      2867 MB

      8 GB

      5734 MB

    Use ossutil to download dump files

    1. Mount the container directory to an NAS directory. For more information, see Configure NAS storage.

    2. Configure JVM options.

      The /home/admin/nas container directory is mounted on an NAS directory and stores dump files.

      -Xms2048m -Xmx2048m -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/home/admin/nas/gc-${POD_IP}-$(date '+%s').log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/admin/nas/dump-${POD_IP}-$(date '+%s').hprof

    3. If OOM errors occur in your application, dump files are generated and stored in the /home/admin/nas directory. You can use the ossutil tool to download the dump files to your on-premises machine and analyze the dump files. For more information, see Upload and download logs to check the health status of applications.

    FAQ

    What do I do if the 137 exit code appears in my container?

    If the used container memory reaches the upper limit, OOM errors may occur in your container. This may cause the container to be forcibly shut down. In this case, if the upper limit of the JVM heap size is not reached, no dump files are generated. To reserve sufficient memory to run system components, we recommend that you decrease the upper limit of the JVM heap size.m_exitcode_137

    What do I do if an OOM occurs and no dump files are generated?

    If the OOM Killer mechanism is triggered, OOM errors may not occur in the JVM. In this case, no dump files are generated. You can use the following methods to prevent the issue:

    • For Java applications, you can decrease the JVM heap size. For more information, see the related instructions in this topic.

    • For non-Java applications, you can adjust the instance specifications to ensure sufficient memory resources. For more information, see Change the instance specifications of an application.

    Can I set the heap size and the memory size to the same value?

    No, you cannot set the heap size and the memory size to the same value because of system component overheads. For example, if you use Simple Log Service to collect logs, a small amount of memory is consumed. You must reserve sufficient memory for system components. For more information, see Configure log collection to Simple Log Service.

    What do I do if an error occurs when I set the XX:MaxRAMPercentage option to an integer in JDK 8?

    This is a bug specific to JDK 8. For more information, see Java Bug Database. Sample scenario: If you set the XX:MaxRAMPercentage option to 70 in JDK 8u191, an error may occur when JVM starts up.m_JDK8_bug

    Solutions:

    • Solution 1: Set the -XX:MaxRAMPercentage option to 70.0.

      Note

      If you use the -XX:InitialRAMPercentage or -XX:MinRAMPercentage option, you cannot set the values to integers. Follow Solution 1 to configure the options.

    • Solution 2: Upgrade JDK to version 10 or later.

    Why is the memory usage low when I set the JVM heap size to 6 GB?

    If you configure the -Xms6g -Xmx6g option, the operating system does not immediately allocate 6 GB of physical memory and allocates the physical memory only after the memory is actually used. The memory usage is relatively low when the application is started and increases later.