Simple Application Server:Troubleshoot high CPU utilization or high CPU load on a Linux simple application server

Use of the vmstat command

Run the vmstat command to view the overall statistics about virtual memory, processes, and CPUs in the operating system.

Command syntax

The following code shows the syntax of the vmstat command:

vmstat [-n] [delay [count]] 

Example

Run the following vmstat command to collect statistics about the CPU utilization of each process every 1 second for four times:

vmstat -n 1 4

Sample output:

procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 1  0      0 2684984 310452 2364304    0    0     5    17   19   35  4  2 94  0  0
 0  0      0 2687504 310452 2362268    0    0     0   252 1942 4326  5  2 93  0  0
 0  0      0 2687356 310460 2362252    0    0     0    68 1891 4449  3  2 95  0  0
 0  0      0 2687252 310460 2362256    0    0     0     0 1906 4616  4  1 95  0  0

Fields in the command output:

• r: indicates the number of threads waiting to be processed by the CPU. A CPU can process only one thread at a time. A larger value indicates that the system runs slower.

• us: indicates the percentage of CPU time consumed in user mode. A larger value indicates that more CPU time is consumed by a user process. If the value exceeds 50% for an extended period of time, you must optimize the program algorithm or code.

• sy: indicates the percentage of CPU time consumed in kernel mode.

• wa: indicates the percentage of CPU time spent waiting for I/O operations to complete. If the value is high, the system experiences significant I/O waits. This may be caused by extensive random access to disks or a bottleneck in disk performance.

• id: indicates the percentage of idle CPU time. If the value of this parameter remains 0 and the value of the sy parameter is twice the value of the us parameter, the CPU resources in the system are insufficient.

Use of the top command

The top command is a common performance analytics tool in Linux. The command can display the resource usage of each process in the system in real time.

Command syntax

top [-n] [-d] 

Example

1. Remotely connect to the Linux simple application server. For more information, see Connect to a Linux server.

2. Run the following command to view the resource usage of each process in the system.

   The command collects statistics about the resource usage of each process every 2 seconds for five times.

   top -n 5 -d 2

   Sample output:

   top - 17:27:13 up 27 days,  3:13,  1 user,  load average: 0.02, 0.03, 0.05
   Tasks:  94 total,   1 running,  93 sleeping,   0 stopped,   0 zombie
   %Cpu(s):  0.3 us,  0.1 sy,  0.0 ni, 99.5 id,  0.0 wa,  0.0 hi,  0.0 si,  0.1 st
   KiB Mem:   1016656 total,   946628 used,    70028 free,   169536 buffers
   KiB Swap:        0 total,        0 used,        0 free.   448644 cached Mem
   PID USER      PR  NI    VIRT    RES    SHR S %CPU %MEM     TIME+ COMMAND
   1 root        20   0   41412   3824   2308 S  0.0  0.4   0:19.01 systemd
   2 root        20   0       0      0      0 S  0.0  0.0   0:00.04 kthreadd

   Fields in the command output:

   For CPU utilization and CPU load issues, you need to only focus on the first and third lines of the command output.

   • On the first line, 17:27:13 up 27 days, 3:13, 1 user, load average: 0.02, 0.03, 0.05 is displayed, which indicates the current system time, the system uptime, the number of logon users, and the system load.

   • On the third line, the overall usage of current CPU resources is displayed. The resource usage of each process is displayed below the line.

3. When the top command runs, you can press the following keys to adjust the command output:

   • Press the P key to sort processes in descending order of CPU utilization. This way, you can quickly find the processes that consume a large amount of CPU resources in the system.

   • Press the M key to sort processes by memory usage. If a CPU has multiple cores, press the 1 key to display the load status of each CPU core.

4. Run the ll /proc/PID/exe command to view the program file that corresponds to each process ID (PID).

5. Terminate the processes that consume a large amount of CPU resources.

   1. Enter k.

   2. Enter the PID of the process that you want to terminate and press the Enter key.

      The default PID is the first PID in the command output. If you want to terminate the process whose PID is 23, enter 23 and press the Enter key, as shown in the following figure.

      

   3. After the operation is complete, a message similar to Send pid 23 signal [15/sigterm] appears. Press the Enter key to confirm.