All Products
Search
Document Center

Elastic Compute Service:Test the IOPS performance of an ESSD

Last Updated:Oct 31, 2023

This topic describes how to test the IOPS performance of an enhanced SSD (ESSD). The specifications of the disk and the test conditions affect the test results. If you configure the test conditions as described in the following example to maximize the performance of a multi-core, high-concurrency system, you can obtain an IOPS value of one million when you perform a stress test on the ESSD.

Test conditions

  • Sample operation: random write (randwrite).

  • Image: We recommend that you use a more recent version of a Linux public image provided by Alibaba Cloud, such as CentOS 7.4 64-bit, CentOS 7.3 64-bit, CentOS 7.2 64-bit, or Alibaba Cloud Linux 2.1903 64-bit.

  • Tool: We recommend that you use FIO.

  • Instance type: We recommend that you use ecs.g7se.32xlarge. For more information, see g7se, storage-enhanced general-purpose instance family.

  • ESSD: We recommend that you use an ESSD at performance level 3 (PL3 ESSD). In this example, /dev/your_device is used as the device name of an ESSD. Replace it with the actual device name. For more information, see ESSDs.

Precautions

Warning
  • You can obtain accurate test results by testing raw disks. However, you may destroy the file system structure in a raw disk if you directly test the disk. Before you test a raw disk, we recommend that you back up your data by creating a snapshot for the disk. For more information, see Create a snapshot for a disk.

  • We recommend that you do not test a disk on which the operating system resides or a disk that stores important data. To prevent data loss, we recommend that you use a new Elastic Compute Service (ECS) instance that contains no data for the test.

Procedure

  1. Connect to an ECS instance.
    For more information, see Connect to an instance by using VNC.
  2. Run the following commands in sequence to install libaio and FIO:

    sudo yum install libaio -y
    sudo yum install libaio-devel -y
    sudo yum install fio -y
  3. Run the following command to switch the path:

    cd /tmp
  4. Run the following command to create the test100w.sh script:

    vim test100w.sh
  5. Paste the following content to the test100w.sh script.

    For information about the script content, see the Details about the test100w.sh script section of this topic.

    function RunFio
    {
     numjobs=$1   # The number of test threads. In this example, the value is 10.
     iodepth=$2   # The maximum number of concurrent I/O requests. In this example, the value is 64.
     bs=$3        # The data block size per I/O. In this example, the value is 4k.
     rw=$4        # The read and write policy. In this example, the value is randwrite.
     size=$5
     filename=$6  # The name of the test file. In this example, the value is /dev/your_device.
     nr_cpus=`cat /proc/cpuinfo |grep "processor" |wc -l`
     if [ $nr_cpus -lt $numjobs ];then
         echo "Numjobs is more than cpu cores, exit!"
         exit -1
     fi
     let nu=$numjobs+1
     cpulist=""
     for ((i=1;i<10;i++))
     do
         list=`cat /sys/block/your_device/mq/*/cpu_list | awk '{if(i<=NF) print $i;}' i="$i" | tr -d ',' | tr '\n' ','`
         if [ -z $list ];then
             break
         fi
         cpulist=${cpulist}${list}
     done
     spincpu=`echo $cpulist | cut -d ',' -f 2-${nu}`
     echo $spincpu
     fio --ioengine=libaio --runtime=30s --numjobs=${numjobs} --iodepth=${iodepth} --bs=${bs} --size=${size} --rw=${rw} --filename=${filename} --time_based=1 --direct=1 --name=test --group_reporting --cpus_allowed=$spincpu --cpus_allowed_policy=split
    }
    echo 2 > /sys/block/your_device/queue/rq_affinity
    sleep 5
    RunFio 10 128 4k randwrite 1024g /dev/your_device
  6. Modify the parameters in the test100w.sh script based on the actual conditions.

    • Replace your_device with the actual device name of the ESSD. Example: nvme1n1.

    • Replace 10, 64, 4k, randwrite, and /dev/your_device in RunFio 10 64 4k randwrite /dev/your_device with the actual values.

    • If data loss of the ESSD does not affect your business, you can set filename to a device name. Example: filename=/dev/vdb. Otherwise, set filename to a file directory. Example: filename=/mnt/test.image.

    Warning
    • You can obtain accurate test results by testing raw disks. However, you may destroy the file system structure in a raw disk if you directly test the disk. Before you test a raw disk, we recommend that you back up your data by creating a snapshot for the disk. For more information, see Create a snapshot for a disk.

    • We recommend that you do not test a disk on which the operating system resides or a disk that stores important data. To prevent data loss, we recommend that you use a new Elastic Compute Service (ECS) instance that contains no data for the test.

  7. Run the following command to test the performance of the ESSD:

    sh test100w.sh

    If the IOPS=*** result is displayed in the command output, the performance stress test for the ESSD is complete.ESSD云盘IOPS性能

Details about the test100w.sh script

  • In the script, the following command sets the rq_affinity parameter to 2:

    echo 2 > /sys/block/your_device/queue/rq_affinity

    Value of rq_affinity

    Description

    1

    Indicates that the block device delivers received I/O completion events to the group of the vCPUs that submit the corresponding I/O requests. In scenarios where multiple threads run concurrently, I/O completion events may be delivered only to one vCPU and cause a performance bottleneck.

    2

    Indicates that the block device delivers received I/O completion events to the vCPUs that submit the corresponding I/O requests. In scenarios where multiple threads run concurrently, each vCPU can deliver its maximum performance.

  • The following command runs jobs to bind queues to different CPU cores:

    fio -ioengine=libaio -runtime=30s -numjobs=${numjobs} -iodepth=${iodepth} -bs=${bs} -rw=${rw} -filename=${filename} -time_based=1 -direct=1 -name=test -group_reporting -cpus_allowed=$spincpu -cpus_allowed_policy=split
    Note

    In normal mode, a device has a single request queue. This request queue becomes a performance bottleneck when multiple threads concurrently process I/O requests. In multi-queue mode, a device can have multiple request queues to process I/O requests and deliver the maximum backend storage performance. For example, assume that you have four I/O threads. To make full use of multi-queue mode and improve storage performance, you must bind these threads to the CPU cores that correspond to different request queues.

    Parameter

    Description

    Example

    numjobs

    The number of I/O threads.

    10

    /dev/your_device

    The device name of the ESSD.

    /dev/nvme1n1

    cpus_allowed_policy

    The parameter provided by FIO to bind vCPUs. FIO provides the cpus_allowed_policy and cpus_allowed parameters to bind vCPUs.

    split

    The preceding command runs jobs to bind queues with different queue IDs to different CPU cores. To view the ID of the CPU core to which a queue is bound, perform the following steps:

    • Run the ls /sys/block/your_device/mq/ command. In this command, replace your_device with the actual device name. Example: nvme1n1. This command returns the ID of the queue for an ESSD whose device name is in the /dev/vd* format.

    • Run the cat /sys/block/your_device/mq//cpu_list command. In this command, replace your_device with the actual device name. Example: nvme1n1. This command returns the ID of the CPU core to which the queue for an ESSD is bound. The device name of the ESSD is in the /dev/vd* format.