Memcg global minimum watermark rating - Alibaba Cloud Linux

When latency-sensitive and resource-consuming tasks run on the same host, burst memory allocation by resource-consuming tasks can push free memory to the global minimum watermark (global wmark_min), causing performance jitter in latency-sensitive tasks. The memcg global minimum watermark rating feature lets you assign different watermark thresholds to individual memory cgroups (memcgs), so latency-sensitive tasks get stronger memory protection and resource-consuming tasks are throttled before they destabilize the system.

Supported versions:

OS	Minimum kernel version
Alibaba Cloud Linux 2	`4.19.91-18.al7`
Alibaba Cloud Linux 3	`5.10.134-12.al8`

How it works

The feature exposes the memory.wmark_min_adj interface on each non-root memcg. The interface defaults to 0 (inherited from the parent group when the interface is created) and is not stored in the memcg root directory. The value you write adjusts the effective minimum watermark for that cgroup as a percentage offset from the global watermark range. Valid values: -25 to 50.

Negative value (−25 to −1) — raises protection priority. The offset is applied over the [0, WMARK_MIN] range, lowering the cgroup's effective minimum watermark below global wmark_min. Memory reclamation from this cgroup is deferred, which reduces performance jitter for latency-sensitive tasks. Example: memory.wmark_min_adj=-25 sets the cgroup's WMARK_MIN to WMARK_MIN + (WMARK_MIN - 0) × (−25%).
Positive value (1 to 50) — lowers protection priority. The offset is applied over the [WMARK_MIN, WMARK_LOW] range, raising the cgroup's effective minimum watermark above global wmark_min. Memory reclamation pressure on this cgroup increases, reducing its memcg quality of service (QoS) and preventing it from crowding out higher-priority tasks. Example: memory.wmark_min_adj=50 sets the cgroup's WMARK_MIN to WMARK_MIN + (WMARK_LOW - WMARK_MIN) × 50%.

When the offset watermark is reached, throttling is applied. Throttling duration is linearly proportional to excess memory usage, within the range of 1–1000 microseconds.

Effective value in a multi-level memcg hierarchy

In a multi-level memcg, the kernel computes an effective memory.wmark_min_adj for each node by traversing all ancestor levels and taking the maximum value. Intermediate nodes with the default value of 0 are excluded from this calculation.

Consider the following hierarchy:

The table below shows how configured values map to effective values at each node:

Level	Configured value	Effective value
A	-10	-10
B	-25	-10
C	0	0
D	50	50
E	-25	-10
F	50	50

To read the effective value for a cgroup, run:

cat /sys/fs/cgroup/memory/<Memcg path>/memory.wmark_min_adj

Replace <Memcg path> with the root path of the target memcg.

Configure watermark rating

Warning

Improper modifications to the memcg QoS may cause network performance issues, including network congestion, increased forwarding latency, and packet loss. This affects the quality and availability of your business. Proceed with caution.

Prerequisites

We recommend that you use this feature together with a high global minimum watermark. For example, set global wmark_min to 2 GB or more by writing to /proc/sys/vm/min_free_kbytes.
Run all configuration commands as the root user.

Configure a latency-sensitive task memcg

Set a negative adjustment to lower the effective watermark and raise memory protection priority for the cgroup.

Create a memcg for the latency-sensitive workload:
```
mkdir /sys/fs/cgroup/memory/test-lc
```

Set the watermark adjustment to -25:

echo -25 > /sys/fs/cgroup/memory/test-lc/memory.wmark_min_adj

Configure a resource-consuming task memcg

Set a positive adjustment to raise the effective watermark and lower memory protection priority for the cgroup.

Create a memcg for the resource-consuming workload:
```
mkdir /sys/fs/cgroup/memory/test-be
```

Set the watermark adjustment to 25:

echo 25 > /sys/fs/cgroup/memory/test-be/memory.wmark_min_adj