This topic describes how to use the fio tool on a Lunix instance to test performance of cloud disks and local disks, including IOPS, throughput, and latency.

Prerequisites

A Block Storage device is created and attached to an ECS instance.
Note If you only want to test the performance of a Block Storage device of a specific type, we recommend that you use a newly created pay-as-you-go data disk. You can release the disk at any time after the test is completed.

Background information

You can use other tools to test the performance of Block Storage devices, but the benchmark you obtain may be different. For example, tools such as dd, sysbench, and iometer may be affected by test parameter settings and file systems, which causes inaccurate results. The performance results in this topic are that of a Linux instance tested by using fio. These results are used as performance references for Alibaba Cloud Block Storage products. We recommend that you use the fio tool to test the performance of Block Storage for both Linux and Windows instances.

Warning You can obtain accurate test results by testing raw disk partitions. However, you may destroy the file system structure in a raw disk partition if you test the partition directly. Before you perform such a test, you must create a snapshot of the disk to back up your data. For more information about how to create a snapshot, see Create a snapshot. We recommend that you use newly purchased ECS instances that contain no data for the test to prevent data loss.

Procedure

  1. Remotely connect to an ECS instance. For more information, see Connect to a Linux instance by using the Management Terminal.
  2. Before you test a Block Storage device, ensure that it is 4 KiB aligned.
    sudo fdisk -lu
    If the value of Start in the command output is divisible by 8, the device is 4 KiB aligned. Otherwise, perform 4 KiB alignment before you continue the performance test.
    Device     Boot Start      End  Sectors Size Id Type
    /dev/vda1  *     2048 83886046 83883999  40G 83 Linux
  3. Run the following commands to install the libaio and fio tools:
    sudo yum install libaio -y
    sudo yum install libaio-devel -y
    sudo yum install fio -y
  4. Switch the directory.
    cd /tmp
  5. Run the performance test commands. For more information about the commands, see the following section.

Commands used to test the performance of cloud disks

For more information about the IOPS test methods regarding the enhanced SSD, see Test the IOPS performance of an enhanced SSD.

  • Random write IOPS:
    fio -direct=1 -iodepth=128 -rw=randwrite -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=iotest -name=Rand_Write_Testing
  • Random read IOPS:
    fio -direct=1 -iodepth=128 -rw=randread -ioengine=libaio -bs=4k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=iotest -name=Rand_Read_Testing
  • Sequential write throughput (write bandwidth):
    fio -direct=1 -iodepth=64 -rw=write -ioengine=libaio -bs=1024k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=iotest -name=Write_PPS_Testing
  • Sequential read throughput (read bandwidth):
    fio -direct=1 -iodepth=64 -rw=read -ioengine=libaio -bs=1024k -size=1G -numjobs=1 -runtime=1000 -group_reporting -filename=iotest -name=Read_PPS_Testing
  • Random write latency:
    fio -direct=1 -iodepth=1 -rw=randwrite -ioengine=libaio -bs=4k -size=1G -numjobs=1 -group_reporting -filename=iotest -name=Rand_Write_Latency_Testing
  • Random read latency:
    fio -direct=1 -iodepth=1 -rw=randread -ioengine=libaio -bs=4k -size=1G -numjobs=1 -group_reporting -filename=iotest -name=Rand_Read_Latency_Testingrandwrite -ioengine=libaio -bs=4k -size=1G -numjobs=1 -group_reporting -filename=iotest -name=Rand_Write_Latency_Testing

Commands used to test the performance of local disks

The following test commands are only applicable to local NVMe SSD disks.

  • Random write IOPS:
    fio -direct=1 -iodepth=32 -rw=randwrite -ioengine=libaio -bs=4k -numjobs=4 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/vdx -name=test
  • Random read IOPS:
    fio -direct=1 -iodepth=32 -rw=randread -ioengine=libaio -bs=4k -numjobs=4 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/vdx -name=test
  • Sequential write throughput (write bandwidth):
    fio -direct=1 -iodepth=128 -rw=write -ioengine=libaio -bs=128k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/vdx -name=test
  • Sequential read throughput (read bandwidth):
    fio -direct=1 -iodepth=128 -rw=read -ioengine=libaio -bs=128k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/vdx -name=test
  • Random write latency:
    fio -direct=1 -iodepth=1 -rw=randwrite -ioengine=libaio -bs=4k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/vdx -name=test
  • Random read latency:
    fio -direct=1 -iodepth=1 -rw=randread -ioengine=libaio -bs=4k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/vdx -name=test
  • Sequential write latency:
    fio -direct=1 -iodepth=1 -rw=write -ioengine=libaio -bs=4k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/vdx -name=test
  • Sequential read latency:
    fio -direct=1 -iodepth=1 -rw=read -ioengine=libaio -bs=4k -numjobs=1 -time_based=1 -runtime=1000 -group_reporting -filename=/dev/vdx -name=test

fio parameter values

The following table uses the command for testing random write IOPS (randwrite) as an example to describe the parameter settings in fio commands.
Parameter Description
-direct=1 Indicates that the I/O buffer is ignored during the test and data is written directly.
-iodepth=128 Indicates that when you use asynchronous I/O, a maximum of 128 I/O requests can be made at the same time.
-rw=randwrite Indicates that the read/write policy is random writes. Other valid values for rw:
  • randread (random reads)
  • read (sequential reads)
  • write (sequential writes)
  • randrw (random reads and writes)
-ioengine=libaio Indicates that libaio (the Linux-native asynchronous I/O facility) is used as the testing method. There are two ways for an application to use I/O:
  • Synchronous

    In synchronous I/O mode, a thread sends only one I/O request at a time and waits until the I/O request is completed. In this case, the iodepth value is always smaller than 1 for a single thread. You can increase the iodepth value by using multiple concurrent threads. The iodepth value will reach its upper limit when there are 16 to 32 threads running concurrently.

  • Asynchronous

    In asynchronous I/O mode, a thread uses libaio to send multiple I/O requests at a time and waits until all these I/O requests are completed. Asynchronous I/O reduces the number of interactions and increases interaction efficiency.

-bs=4k Indicates that the size of each block for one I/O is 4 KB. If this parameter is not specified, the default value 4 KB is used.
  • When IOPS is tested, we recommend that you set bs to a small value, such as 4k in this example command.
  • When throughput is tested, we recommend that you set bs to a large value, such as 1024k in other example commands.
-size=1G Indicates that the size of the testing file is 1 GiB.
-numjobs=1 Indicates that the number of testing threads is 1.
-runtime=1000 Indicates that the test duration is 1,000 seconds. If this parameter is not specified, the file of the size specified by -size is written in blocks of the size specified by -bs.
-group_reporting Indicates that in the testing results, the statistics are displayed for a group of threads, not for each individual thread.
-filename=iotest Indicates that the name of the test file is iotest.
-name=Rand_Write_Testing Indicates that the name of the test task is Rand_Write_Testing. You can specify any name you want.