Testing Elastic Block Storage performance helps you understand the capabilities of your storage devices so that you can optimize them for the best performance. Testing a raw disk provides a realistic measure of its performance. This topic describes how to use the open source tool FIO on a Linux system to test key performance metrics of a raw disk, such as input/output operations per second (IOPS), throughput, and latency.
FIO (Flexible I/O Tester) is a powerful, open source I/O performance testing tool. You can use it to run load tests, such as random and sequential reads and writes, on storage devices.
Procedure
Testing a raw disk provides the most realistic measure of its performance. However, running a stress test with FIO directly on a device that contains partitions, a file system, or other data can cause file system errors and data loss. Before you start the test, create a snapshot to back up your data. For more information, see Create a snapshot.
To prevent data loss, do not test the system disk or a data disk that contains data. Test the storage performance on a new, uninitialized, and empty data disk.
The performance test results are from a staging environment and are for reference only. In a production environment, disk performance may vary based on factors such as the network environment and concurrent access.
After the test is complete, manage the new disk as follows:
If you want to keep the disk, re-initialize the data disk before you use it.
If you do not need the disk, detach the data disk and then release the disk.
This topic uses a public image of Alibaba Cloud Linux 3.2104 LTS 64-bit as an example. The steps may vary depending on your environment.
Connect to an ECS instance.
For more information, see Connect to a Linux instance using Workbench.
Run the following command to retrieve the device names of your Elastic Block Storage devices.
sudo fdisk -lu
The preceding figure shows that the instance has three Elastic Block Storage devices: the system disk /dev/vda, and the data disks /dev/vdb and /dev/vdc.Run the following command to check for partitions and file systems on the Elastic Block Storage devices.
sudo blkid
The preceding figure shows that the Elastic Block Storage devices /dev/vda and /dev/vdb have partitions and file systems. The command output does not contain information about /dev/vdc, which indicates that it has no partitions or file systems.
WarningRunning a stress test with FIO directly on a device that contains partitions, a file system, or other data can cause file system errors and data loss. If your data disk has partitions and a file system, create a new, empty data disk for testing:
Create a pay-as-you-go disk with the same configuration and attach it to the instance for testing. For more information, see Create an empty data disk.
After testing is complete, you can release the instance. For more information, see Release an instance.
Before you test the performance of the Elastic Block Storage device, back up its data to prevent data loss. For more information, see Create a snapshot.
NoteSnapshots are a billable service. For more information, see Snapshot billing.
Run the following command to install the libaio library and the FIO tool. Select the command that corresponds to your operating system.
Alibaba Cloud Linux 2/3, CentOS 6 and later
NoteCentOS 6 reached end of life (EOL). In accordance with Linux community rules, all content was removed from the following CentOS 6 repository address: http://mirror.centos.org/centos-6/. If you continue to use the default CentOS 6 repository on Alibaba Cloud, an error is reported. To use specific installation packages of CentOS 6, change the CentOS 6 repository address. For more information, see How do I change CentOS 6 repository addresses?
sudo yum install libaio libaio-devel fio -yDebian 9 and later, Ubuntu 14 and later
ImportantBecause Debian 9 and Debian 10 have reached their end of life (EOL), if you use an ECS instance that runs Debian 9 or Debian 10, you must switch the source address first. For more information, see Switch the source address for an operating system after its EOL.
sudo apt-get update sudo apt-get install libaio* fio -yChange the path:
cd /tmpRun the performance test commands. For more information, see the following sections:
For commands to test disks, see Disk performance test commands.
For commands to test local disks, see Local disk performance test commands.
Analyze the test results. The values vary for different disks. The values in the following figures are for reference only.
For IOPS results, view the value of
IOPS=***, as shown in the following figure:
For throughput results, view the value of
BW=***, as shown in the following figure:
For latency results, view the value of lat (usec), as shown in the following figure:
Disk performance test commands
Running a stress test with FIO directly on a device that contains partitions, a file system, or other data can cause file system errors and data loss. If your data disk has partitions and a file system, create a new, empty data disk for testing:
Create a pay-as-you-go disk with the same configuration and attach it to the instance for testing. For more information, see Create an empty data disk.
After the test is complete:
If you want to keep the disk, re-initialize the data disk.
If you do not need the disk, detach and then release the disk.
The parameter values in the following sample commands are for reference only. Replace /dev/your_device with the name of the Elastic Block Storage device that you want to test, which you retrieved in Step 2. For example, if the device name of the disk to test is /dev/vdb, replace /dev/your_device with /dev/vdb in the sample commands. For more information about FIO parameters, see FIO parameter description.
Test the random write IOPS of a disk:
sudo fio -direct=1 -iodepth=128 -rw=randwrite -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=Rand_Write_TestingTest the random read IOPS of a disk:
sudo fio -direct=1 -iodepth=128 -rw=randread -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=Rand_Read_TestingTest the sequential write throughput of a disk:
sudo fio -direct=1 -iodepth=64 -rw=write -ioengine=libaio -bs=1024k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=Write_PPS_TestingTest the sequential read throughput of a disk:
sudo fio -direct=1 -iodepth=64 -rw=read -ioengine=libaio -bs=1024k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=Read_PPS_TestingTest the random write latency of a disk:
sudo fio -direct=1 -iodepth=1 -rw=randwrite -ioengine=libaio -bs=4k -size=1G -numjobs=1 -group_reporting -filename=/dev/your_device -name=Rand_Write_Latency_TestingTest the random read latency of a disk:
sudo fio -direct=1 -iodepth=1 -rw=randread -ioengine=libaio -bs=4k -size=1G -numjobs=1 -group_reporting -filename=/dev/your_device -name=Rand_Read_Latency_Testing
For more information about how to test different disks, see Test the IOPS performance of an ESSD.
Local disk performance test commands
Running a stress test with FIO directly on a device that contains partitions, a file system, or other data can cause file system errors and data loss. If your data disk has partitions and a file system, create a new instance with local disks for testing.
Local disks can be created only together with an instance. To test a local disk, create a new instance with the same configuration. For more information, see Create an instance using the wizard.
After the test is complete, release the instance if you no longer need it. For more information, see Release an instance.
The following test commands apply to NVMe SSD local disks and SATA HDD local disks. The parameter values in the following sample commands are for reference only. Replace /dev/your_device with the name of the local disk that you want to test, which you retrieved in Step 2. For example, if the device name of the local disk to test is /dev/vdb, replace /dev/your_device with /dev/vdb in the sample commands. For more information about FIO parameters, see FIO parameter description.
Test the random write IOPS of a local disk:
sudo fio -direct=1 -iodepth=32 -rw=randwrite -ioengine=libaio -bs=4k -numjobs=4 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=testTest the random read IOPS of a local disk:
sudo fio -direct=1 -iodepth=32 -rw=randread -ioengine=libaio -bs=4k -numjobs=4 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=testTest the sequential write throughput of a local disk:
sudo fio -direct=1 -iodepth=128 -rw=write -ioengine=libaio -bs=128k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=testTest the sequential read throughput of a local disk:
sudo fio -direct=1 -iodepth=128 -rw=read -ioengine=libaio -bs=128k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=testTest the random write latency of a local disk:
sudo fio -direct=1 -iodepth=1 -rw=randwrite -ioengine=libaio -bs=4k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=testTest the random read latency of a local disk:
sudo fio -direct=1 -iodepth=1 -rw=randread -ioengine=libaio -bs=4k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=testTest the sequential write latency of a local disk:
sudo fio -direct=1 -iodepth=1 -rw=write -ioengine=libaio -bs=4k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=testTest the sequential read latency of a local disk:
sudo fio -direct=1 -iodepth=1 -rw=read -ioengine=libaio -bs=4k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/your_device -name=test
FIO parameter description
The following table describes the FIO parameters used in the test commands.
Parameter | Description |
direct | Specifies whether to use direct I/O. The default value is 1.
|
iodepth | The I/O queue depth for the test. For example, |
rw | The read and write policy for the test. You can set this parameter to one of the following values:
|
ioengine | The I/O engine that FIO uses for the test. We recommend that you select libaio because it is suitable for most application scenarios. For more information about other options, see the official FIO documentation. |
bs | The block size for each I/O unit. The default value is 4 KiB. You can specify separate values for read and write operations in the format of read,write. If a value is left empty, the default value is used. |
size | The size of the test file. FIO reads or writes the specified file size and then stops the test, unless restricted by other options such as runtime. If you do not specify this parameter, FIO uses the full size of the specified file or device. You can also specify the size as a percentage from 1 to 100. For example, if you set size to 20%, FIO uses 20% of the space of the specified file or device. |
numjobs | The number of concurrent threads for the test. The default value is 1. |
runtime | The duration of the test. If you do not specify this parameter, FIO continues to run until it finishes reading or writing the file of the size specified by the size parameter in blocks of the size specified by the bs parameter. |
group_reporting | The display mode for the test results. If you specify this parameter, the results show aggregated statistics for each process instead of for each task. |
filename | The path of the object to test. The path can be the device name of a disk or a file path. In this topic, FIO tests are performed on an entire disk that does not have a file system (a raw disk). To prevent data on other disks from being damaged, the example path is /dev/your_device. Replace it with the actual path. |
name | The name of the test task. You can set this to any name. For example, Rand_Write_Testing. |
For more information about the parameters, see the FIO man page.