The memcg global minimum watermark rating feature is supported in Alibaba Cloud Linux 2 starting with the kernel version
4.19.91-18.al7 and Alibaba Cloud Linux 3. This topic describes the feature.
Global memory reclamation has a great impact on system performance in Linux kernels. When latency-sensitive tasks and resource-consuming tasks are deployed together, resource-consuming tasks request a large amount of memory instantly, which causes the free memory of the system to reach the global minimum watermark (global wmark_min). As a result, direct memory reclamation is enabled for all system tasks, which causes performance jitters for latency-sensitive tasks. Neither global kswapd backend reclamation or memcg backend reclamation can resolve this problem.
In this context, Alibaba Cloud Linux provides the memcg global minimum watermark rating feature. The global minimum watermark of resource-consuming tasks is increased to trigger direct memory reclamation. The global minimum watermark of latency-sensitive tasks is decreased to prevent direct memory reclamation. When a resource-consuming task requests a large amount of memory, the global minimum watermark is increased to throttle resources used for the task for a short period to prevent direct memory reclamation for latency-sensitive tasks. After a specified amount of memory is reclaimed by means of global kswapd backend reclamation, the resource-consuming task is no longer throttled.
The interface that implements the memcg global minimum watermark rating feature is memory.wmark_min_adj. The value of this interface indicates the percent of adjustment over the global minimum watermark. Valid values: -25 to 50.
This interface inherits a value of 0 from its parent group when the interface is created. Therefore, the default value is 0.
A negative value is a percent of adjustment over the
[0, WMARK_MIN]range, where
WMARK_MINis the value of global wmark_min. Example:
memory.wmark_min_adj=-25, memcg WMARK_MIN is "WMARK_MIN + (WMARK_MIN - 0) * (-25%)"Note
A negative value also indicates a decrease of the global minimum watermark. This can increase the memcg quality of service (QoS) of latency-sensitive tasks.
A positive value is a percent of adjustment over the
[WMARK_MIN, WMARK_LOW]range, where
WMARK_LOWare values of global wmark_min and global wmark_low. Example:
memory.wmark_min_adj=50, memcg WMARK_MIN is "WMARK_MIN + (WMARK_LOW - WMARK_MIN) * 50%"Note
A positive value also indicates an increase of the global minimum watermark. This can decrease the memcg QoS of resource-consuming tasks.
When the offset global minimum watermark is triggered, throttling is performed, and the throttling time is linearly proportional to the excess memory usage. Valid values of the throttling time: 1 to 1000. Unit: ms.
This interface file is not stored in the memcg root directory.
A multi-level memcg contains
effective memory.wmark_min_adj, which is the final effective value of memory.wmark_min_adj. The values of memory.wmark_min_adj at all levels are traversed to obtain the maximum value. Intermediate nodes that have 0 as the default value are excluded. The following hierarchy provides an example:
The following table describes the mapping between the values of memory.wmark_min_adj at each level and the final effective value.
Value at each level
Final effective value
The value displayed in the output of the
cat /sys/fs/cgroup/memory/<Memcg path>/memory.wmark_min_adjcommand is the final effective value. In the command,
<Memcg path>indicates the root path of the memcg.
We recommend that you use the global minimum watermark rating feature together with global wmark_min. For example, you can set global wmark_min to 2 GB or more in /proc/sys/vm/min_free_kbytes.
Improper modifications to the memcg QoS may cause network performance degradation, such as network congestion, increased forwarding latency, and packet loss. This negatively affects the quality and availability of your business. Proceed with caution.
Example 1: Configure global minimum watermark rating for the memcg of latency-sensitive tasks.
mkdir /sys/fs/cgroup/memory/test-lccommand to create a test file.
echo -25 > /sys/fs/cgroup/memory/test-lc/memory.wmark_min_adjcommand to write the value of
-25to the interface to increase the memcg QoS of latency-sensitive tasks.
Example 2: Configure global minimum watermark rating for the memcg of resource-consuming tasks.
mkdir /sys/fs/cgroup/memory/test-becommand to create a test file.
echo 25 > /sys/fs/cgroup/memory/test-be/memory.wmark_min_adjcommand to write the value of
25to the interface to decrease the memcg QoS of resource-consuming tasks.