cgroup v2 uses a unified hierarchy and an improved memory management mechanism, and is compatible with enhanced features in future Kubernetes versions. Migrating your nodes to a newer operating system that enables cgroup v2 by default improves cluster stability and resource management efficiency.
Cgroup version guide
The Linux kernel provides two versions of cgroups: cgroup v1 and cgroup v2. Cgroups limit, account for, and isolate the physical resources used by process groups, such as CPU, memory, and I/O. cgroup v2 is the next-generation cgroup interface. It resolves the multiple controller hierarchy issues of cgroup v1 and provides more consistent resource control. The two versions differ in their common interfaces and how subsystems are organized. Applications that directly access the cgroup file system must be updated for compatibility.
For more information, see Differences between cgroup v1 and cgroup v2.
Starting with version 1.31, Kubernetes has moved cgroup v1 support to maintenance mode. Therefore, migrating to cgroup v2 is recommended. The main advantages of cgroup v2 include the following:
Improved stability: cgroup v2 uses a unified memory accounting model to effectively manage Page Cache. This resolves an issue in cgroup v1 where applications with high disk I/O preempt memory, which causes other applications to be killed by out-of-memory (OOM) events.
Unified hierarchy: All resource controllers, such as CPU and memory, are organized in a single hierarchy. This avoids the configuration conflicts and management complexity of multiple hierarchies in cgroup v1 and simplifies resource views and policy definitions.
Enhanced resource observability: cgroup v2 introduces Pressure Stall Information (PSI). PSI quantifies the time an application is blocked while waiting for CPU, memory, or I/O resources. This provides fine-grained metrics for analyzing performance bottlenecks.
Check the cgroup version of a node
Before you migrate, log on to the node and run the following command to check the current cgroup version.
# Run this command after logging on to the target node
stat -fc %T /sys/fs/cgroup/
# Expected output:
# cgroup2fs --> Indicates cgroup v2
# tmpfs --> Indicates cgroup v1
Migration steps
Node level: Change the operating system
The cgroup version of an ACK cluster node is determined by its operating system. To migrate, you must change the operating system at the node pool level to a newer version that supports cgroup v2. In ACK, operating systems that use cgroup v2 by default include Alibaba Cloud Linux 3.2104 LTS 64-bit Container-Optimized Edition, Alibaba Cloud Linux 4 LTS 64-bit Container-Optimized Edition, ContainerOS 3.3 and later, RHEL 9 and later, and Ubuntu 22 and later.
For more information, see Change the operating system.
Application level: Ensure workload compatibility
Because the file system structures and parameter names of cgroup v1 and v2 are incompatible, any application that directly reads the /sys/fs/cgroup path must be verified or upgraded to support cgroup v2.
Category | Description |
Java applications |
|
Go applications | If you use the uber-go/automaxprocs package, upgrade it to v1.5.1 or later. |
cAdvisor | If you use cAdvisor as a standalone DaemonSet to monitor pods and containers, update it to v0.43.0 or later. |
Nginx Ingress | Older versions might fail to correctly parse CPU core information in cgroup v2. This can lead to excessive memory usage and trigger an OOM event. Upgrade to v1.11.2 or later. For more information, see GitHub Issue #9665. When you upgrade the ACK Nginx Ingress Controller, see Upgrade the Nginx Ingress Controller component. |
Other applications and components to note
Third-party monitoring and APM agents: Tools such as Prometheus Node Exporter, Datadog Agent, and SkyWalking rely on the cgroupfs data source to collect node or container metrics. Versions that are not compatible with cgroup v2 may cause missing or abnormal data. Check the official documentation and upgrade to a version that explicitly supports cgroup v2.
Security and auditing tools: Tools such as Falco and Sysdig use cgroup information to associate events with their sources. If a component version is incompatible, detection rules might fail or generate false positives. Upgrade to a compatible version and verify the effectiveness of the rules in a staging environment.
Performance-sensitive applications and custom scripts: Some application startup scripts read cgroup files for automatic tuning, such as setting the number of threads based on the CPU quota. This logic fails under cgroup v2 because of path changes. Review the relevant scripts and update them to read data in a way that is compatible with cgroup v2.
Recommendations for production environments
Application compatibility
Review your custom applications or scripts to ensure they do not depend on the cgroup v1 file system, such as
cpu.cfs_quota_us. Because the file system interface is incompatible, this logic will fail under cgroup v2 and must be updated.Custom node configurations
Changing the operating system resets the node. All custom modifications must be persisted using node pool features. Otherwise, the modifications will be lost after the node is rebuilt. For related configuration methods and notes, see:
Monitoring and alerting: Enable Alibaba Cloud Prometheus Monitoring to continuously observe the overall health of the cluster and the resource usage trends of key containers. This helps you promptly detect any abnormalities.