When multiple threads or processes concurrently read the /proc/cpuinfo file, the system sends inter-processor interrupt (IPI) requests to all CPU cores to obtain the most recent frequency-related data. This generates a large number of waiting IPI requests to form an IPI storm, which reduces the system performance and response speed. To resolve this issue, Alibaba Cloud optimizes the related implementation in Alibaba Cloud Linux 3 with kernel version 5.10.134-17.al8 and later in the x86 architecture.
Optimization solution
To resolve the IPI storm issue that occurs when all threads or processes concurrently obtain CPU frequency data, the threads and processes need to obtain a copy of the most recent backup data. Then, an atomic exchange (
atomic_xchg()) is performed to check and update field values in the backup data. The validity period of the backup data is approximately 20 milliseconds. During the process of obtaining CPU frequency data, a new call of thesmp_call_function_single()function can be made to request updated data only after the validity period ends. This prevents the threads or processes from repeatedly sending a large number of IPI requests to the same CPU within a specific period of time.When the system reads the
/proc/cpuinfofile to obtain CPU frequency data, the system sends IPI requests to all CPUs, which forcefully wakes up the CPUs in theIdlestate. During the CPU wake-up process, the CPU frequency may change, which is determined by the dynamic scheduling algorithm. To improve system efficiency, optimization is performed. When the CPUs are in theIdlestate, the NOHZ feature is enabled in Alibaba Cloud Linux 3 to allow the idle CPUs to return base frequency values without the need to wait for IPI requests sent by the system. This prevents CPU frequency deviations due to CPU status changes.
Benefits and disadvantages
Benefits
Reduced resource consumption: The improper use of IPI requests is prevented, and the IPI requests that wake up idle CPUs are reduced, which effectively reduces the processing time.
Optimized overall performance: In the multi-core system environment, the impact on other CPU management activities such as task scheduling is reduced, which helps maintain the efficient and stable performance of the system.
Disadvantages
Although the optimization solution can reduce resource consumption and improve the overall performance, the optimization solution causes the frequency of CPUs in the
Idlestate that the system obtains by reading the/proc/cpuinfofile to be lower than the actual frequency of CPUs in theIdlestate, which causes the fake "frequency reduction" issue. The accurate CPU frequency data can be obtained by using theturbostattool or other tools to read the MSR.