What do I do if the disk I/O load is high on a Linux instance? -

When you use Linux Elastic Compute Service (ECS) instances, the high disk I/O load issue may cause instance stuttering or slow file read/write operations. This topic describes the causes of and solutions to the issue.

Problem description

When you use a Linux ECS instance, the following issues may occur:

System stuttering, slow file read/write operations, application performance degradation, or slow internal service response occurs.
The disk I/O load statistics in the ECS console indicate that the disk I/O load is too high. For example, the I/O load is considered high if the current read/write rate is greater than or equal to 80% of the disk I/O performance metric.
You are alerted of the message that the disk I/O load exceeds the threshold.

Possible causes

High disk I/O load issue may occur due to the following reasons:

An abnormal process or service consumes many disk I/O resources.
Business programs and scenarios require high disk I/O that exceeds the disk I/O performance capacity of the Windows ECS instance.

Troubleshoot the issues

To identify and troubleshoot the issue, perform the following steps:

Use iostat to query the overall disk I/O load

The iostat command is used to monitor I/O performance in Linux. You can use iostat to query the overall disk I/O load.

Run the following command to view the disk I/O load in the operating system:
```
sudo iostat -d -m 3 5
```
Handle the "iostat: command not found" error
Most Linux distributions have the iostat tool pre-installed in the Sysstat tool collection, without the need for manual installation. If the iostat command is not found, run a command to install the iostat tool based on your Linux distribution.
Alibaba Cloud Linux, CentOS, or Fedora
sudo yum install -y sysstat
Ubuntu or Debian
sudo apt install -y sysstat
openSUSE
sudo zypper install -y sysstat
Note
- -d: displays only device statistics, not CPU usage.
- -m: displays statistics in MB.
- 3: The data collection interval. Unit: seconds.
- 5: The number of times of data collection.
The following command output is returned:
```
Linux 5.10.134-18.al8.x86_64 (iZbpxxxxxxxxxxxxxxxaqhZ)  04/07/2025      _x86_64_        (4 CPU)

Device             tps    MB_read/s    MB_wrtn/s    MB_read    MB_wrtn
vda             145.67         0.00       126.81          0        380
vdb               0.00         0.00         0.00          0          0

Device             tps    MB_read/s    MB_wrtn/s    MB_read    MB_wrtn
vda             161.33         0.00       127.24          0        381
vdb               0.00         0.00         0.00          0          0
...
```
Take note of the following parameters in the command output. For information about other parameters, run the man iostat command.
- Device: The name of the disk.
- tps: The number of I/O requests per second.
- MB_read/s: The file read speed per second.
- MB_wrtn/s: The file write speed per second.
- MB_read: The size of files read.
- MB_wrtn: The size of files written.

Important

If the actual disk I/O performance is significantly lower than the performance metrics released by Alibaba Cloud for the disk category, we recommend that you check whether the disk partitions are 4K aligned. This helps you quickly identify performance overhead issues caused by improper alignment configuration.
For information about block storage performance, see Block storage performance.

Use iotop to check the disk I/O load of processes

The iotop tool is a Linux command and a top-like utility used to monitor disk I/O usage. You can use iotop to check the disk I/O load of individual processes.

Run a command to install iotop based on the Linux distribution.
Alibaba Cloud Linux, CentOS, or Fedora
```
sudo yum install -y iotop
```
Ubuntu or Debian
```
sudo apt install -y iotop
```
openSUSE
```
sudo zypper install -y iotop
```
Run the following command to check the disk I/O load:
```
sudo iotop -P -k -d 3
```
Note
- -P: displays the process ID (PID).
- -k: displays in the unit of KB.
- -d: specifies the data collection interval. Unit: seconds. Default value: 1.
The following command output is returned. To exit the program, press the q key.
```
Total DISK READ :       0.00 K/s | Total DISK WRITE :   80373.45 K/s
Actual DISK READ:       0.00 K/s | Actual DISK WRITE:  127781.85 K/s
    PID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND                                                                                                                  
  17250 be/4 root        0.00 K/s 80354.87 K/s  0.00 % 95.99 % dd if=/dev/zero of=/mnt/dev_vdb/swap bs=1M count=10240
  15192 be/3 root        0.00 K/s    6.64 K/s  0.00 % 60.56 % [jbd2/vdb1-8]
  17152 be/4 root        0.00 K/s    0.00 K/s  0.00 % 59.55 % [kworker/u8:0
...
```
Take note of the following parameters. For information about other parameters, run the iotop -h command.
- PID: the PID.
- DISK READ: the read I/O speed of the process.
- DISK WRITE: the write I/O speed of the process.
- SWAPIN: the percentage of the CPU time that the process spends waiting for memory pages to be swapped in from the swap space.
- IO>: the percentage of the CPU time that the process spends waiting for I/O, including swap-in and disk I/O waiting.
- COMMAND: the command name of the process.

Note

If the kjournald process consumes more disk I/O resources than other processes, we recommend that you change the journal size of the corresponding partition. For more information, see How do I handle the issue when the kjournald process consumes a large number of disk I/O resources?

Resolve the issues

The following table describes the common causes of and solutions to high disk I/O load issues.

Problem description	Cause	Solution
An abnormal user program or process consumes a large number of disk I/O resources for a long time.	The program or process is abnormal and consumes excessive disk I/O resources during the runtime.	Use the iotop tool to locate the PID of the program or process that consumes a large number of disk I/O resources, and end the process by running the `sudo kill -9 <PID>` command. Warning Before you end a process, check the program or process information to prevent business interruptions due to accidental operations. Proceed with caution.
A normal user program or process consumes a large number of disk I/O resources for a long time.	The program or process that consumes excessive disk I/O resources during the runtime is normal.	If the disk encounters an I/O performance bottleneck, perform one of the following operations based on your actual business scenario: Change the disk category: If a standard SSD encounters a disk I/O bottleneck, change the disk category to Enterprise SSD (ESSD) to improve disk performance. Change the disk performance level: If an ESSD encounters a disk I/O bottleneck, upgrade the performance level of the disk. For more information, see Modify the performance level of an ESSD. Modify the disk performance configuration: If an ESSD AutoPL disk encounters a disk I/O bottleneck, modify the disk performance configuration. For more information, see Modify the performance configuration of an ESSD AutoPL disk. Upgrade the instance type: If you cannot upgrade the disk specifications due to instance type limits, upgrade the instance type. For more information, see Change the instance type.
No individual program or process consumes a large number of disk I/O resources. A single program or process occasionally has high disk I/O usage within a short period of time.	The services on the instance require higher disk I/O performance than that the current disk provides.

FAQ

How do I check whether the disk partitions are 4K aligned?

Perform the following steps to check whether the disk partitions are 4K aligned. In this example, the /dev/vdb1 partition is used. Replace the value with the actual partition name.

Note

The 4K alignment feature aligns the start positions of partitions and the logical cluster of the file system with the physical sector size of the hard disk (4 KB). This improves disk I/O performance.

Run the following command to check if the device partition is 4K aligned:
```
sudo parted /dev/vdb1 align-check optimal 1
```
The following command output indicates that the partition is already 4K aligned. In this case, you can skip the following steps and proceed with handling high disk I/O load issues.
```
1 aligned
```
If the partition is not 4K aligned, manually perform 4K alignment. For more information, see What do I do when a GPT partition is created on a data disk but is misaligned?
After you complete 4K alignment, check whether the read/write I/O performance metrics of disks are consistent with those released by Alibaba Cloud for the disk category.

How do I handle the issue when the kjournald process consumes a large number of disk I/O resources?

Problem description: The iotop tool detects that the kjournald process occupies a large number of disk I/O resources.
Cause: The Total journal size value of the Ext3 file system is set too small.
Note
kjournald is the core log management process of the Ext3 file system. kjournald logs and synchronizes I/O operations. When data is continuously written to the Ext3 file system, the kjournald process accumulates the journal space usage. If the journal size reaches the preset Total journal size value, the system triggers a forced journal flush. At this time, the process blocks subsequent write requests until journal synchronization is complete. This may lead to intensive CPU contention, memory usage spikes, and overall I/O performance degradation.
Solution:
- Use Workbench to connect to a Linux instance over SSH.
- Run the following command to check the journal size of the corresponding partition. In the following example, the /dev/vdb1 partition is used. Replace the value with the actual partition name.
```
sudo dumpe2fs /dev/vdb1 | grep -i Journal
```
  The following command output indicates that Total journal size of the /dev/vdb1 partition is 64 MB.
```
dumpe2fs 1.46.0 (29-Jan-2020)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype sparse_super large_file
Journal inode:            8
Journal backup:           inode blocks
Journal features:         (none)
Total journal size:       64M
Total journal blocks:     16384
Journal sequence:         0x00000001
Journal start:            0
```
- Run the following command to query the current mount information of the partition:
```
sudo mount | grep /dev/vdb1
```
  The following command output is returned. Record the information that you can use to remount the partition.
```
/dev/vdb1 on /mnt/dev_vdb type ext4 (rw,relatime)
```
- Run the following command to unmount the partition:
```
sudo umount /dev/vdb1
```
  Note
  If the umount: /mnt/dev_vdb: target is busy error message appears, the disk is in use. In this case, end the process that uses the disk.
- Run the following command to change the journal size to 400 MB. Set a proper value for Total journal size based on the factors such as actual business scenarios and block storage device parameters.
```
sudo mke2fs -J size=400 /dev/vdb1
```
- After the parameter value is changed, run the following command to remount the partition:
```
sudo mount /dev/vdb1 /mnt/dev_vdb/
```

:What do I do if the disk I/O load is high on a Linux instance?