The disk configuration and actual environment used will affect the test result. You can configure the environment to perform a performance stress test on an enhanced SSD and obtain the one million IOPS result as shown in this example.

Background information

Testing conditions:

  • Sample operation: random write (randwrite).
  • Image: We recommend that you use the latest version of a Linux image provided by Alibaba Cloud, such as CentOS 7.4 64-bit, CentOS 7.3 64-bit, CentOS 7.2 64-bit, or Aliyun Linux 2.1903 64-bit.
  • Tool: We recommend that you use fio.
  • Instance type: We recommend that you use ecs.g5se.18xlarge.
  • ESSD: We recommend that you use a PL3 enhanced SSD. In this example, the device name of the enhanced SSD is /dev/vdb. For more information, see Enhanced SSDs.
    Warning You can obtain accurate test results by testing raw disk partitions. However, you may destroy the file system structure in a raw disk partition if you test the partition directly. Before you perform such a test, you must create a snapshot of the disk to back up your data. For more information about how to create a snapshot, see Create a snapshot. We recommend that you use newly purchased ECS instances that contain no data for the test to prevent data loss.

Procedure

  1. Remotely connect to an ECS instance. For more information, see Connect to a Linux instance by using the Management Terminal.
  2. Run the following commands to install the libaio and fio tools:
    sudo yum install libaio -y
    sudo yum install libaio-devel -y
    sudo yum install fio -y
  3. Switch the directory.
    cd /tmp
  4. Create the test100w.sh script.
    vim test100w.sh
  5. Paste the following code to the test100w.sh script.
    function RunFio
    {
     numjobs=$1   # The number of the threads to be tested. In this example, the value is 10.
     iodepth=$2   # The maximum number of concurrent I/O requests. In this example, the value is 64.
     bs=$3        # The size of the data block per I/O. In this example, the value is 4k.
     rw=$4        # The read and write policy. In this example, the value is randwrite.
     filename=$5  # The name of the file to be tested. In this example, the value is /dev/vdb.
     nr_cpus=`cat /proc/cpuinfo |grep "processor" |wc -l`
     if [ $nr_cpus -lt $numjobs ];then
         echo "Numjobs is more than cpu cores, exit!"
         exit -1
     fi
     let nu=$numjobs+1
     cpulist=""
     for ((i=1;i<10;i++))
     do
         list=`cat /sys/block/vdb/mq/*/cpu_list | awk '{if(i<=NF) print $i;}' i="$i" | tr -d ',' | tr '\n' ','`
         if [ -z $list ];then
             break
         fi
         cpulist=${cpulist}${list}
     done
     spincpu=`echo $cpulist | cut -d ',' -f 2-${nu}`
     echo $spincpu
     fio --ioengine=libaio --runtime=30s --numjobs=${numjobs} --iodepth=${iodepth} --bs=${bs} --rw=${rw} --filename=${filename} --time_based=1 --direct=1 --name=test --group_reporting --cpus_allowed=$spincpu --cpus_allowed_policy=split
    }
    echo 2 > /sys/block/vdb/queue/rq_affinity
    sleep 5
    RunFio 10 64 4k randwrite /dev/vdb
  6. You need to modify the parameter settings in the test100w.sh script based on your environment.
    • Set vdb to the actual device name of the enhanced SSD.
    • Modify 10, 64, 4k, randwrite, and /dev/vdb in RunFio 10 64 4k randwrite /dev/vdb.
    • If you choose to proceed with this operation, set filename to a device name such as [/dev/vdb]. If you do not want to risk data loss, set filename to a file path such as [/mnt/test.image].
  7. Test the performance of the enhanced SSD.
    sh test100w.sh
    When the IOPS=*** appears, it indicates that the performance test of the enhanced SSD is completed.Test the performance of the enhanced SSD

Details of the test100w.sh script

  • In the script, the following command sets the rq_affinity parameter to 2:
    echo 2 > /sys/block/vdb/queue/rq_affinity
    Value of rq_affinity Description
    1 Indicates that the block device forwards the received I/O Completion events to the vCPU group that submits the corresponding I/O requests. When multiple threads concurrently process I/O requests, the I/O Completion events may run on the same vCPU, which results in a performance bottleneck.
    2 Indicates that the block device forwards the received I/O Completion events to the vCPU that submits the corresponding I/O requests. The performance of each vCPU is fully delivered when multiple threads concurrently process I/O requests.
  • The following command attaches jobs to different CPU cores.
    fio -ioengine=libaio -runtime=30s -numjobs=${numjobs} -iodepth=${iodepth} -bs=${bs} -rw=${rw} -filename=${filename} -time_based=1 -direct=1 -name=test -group_reporting -cpus_allowed=$spincpu -cpus_allowed_policy=split
    Note Typically, a device has only one request queue. This unique request queue becomes a performance bottleneck when multiple threads concurrently process I/O requests. In Multi-Queue mode, one device can have multiple request queues that process I/O requests, delivering full performance of the backend storage service. If you have four I/O threads, you must attach them to the CPU cores that correspond to different request queues. This allows you to leverage the full capabilities of the Multi-Queue mode to improve storage performance.
    Parameter Description Example
    numjobs The number of I/O threads. 10
    /dev/vdb The device name of the enhanced SSD. /dev/vdb
    cpus_allowed_policy fio provides the cpus_allowed_policy and cpus_allowed parameters to bind vCPUs. split
    The preceding command runs multiple jobs that are bound to different CPU cores and correspond to different queue IDs. To view the value of cpu_core_id that is bound to a queue ID, perform the following steps:
    • Run the ls /sys/block/vd*/mq/ command to view the queue ID of the enhanced SSD whose device name is in /dev/vd* format.
    • Run the cat /sys/block/vd*/mq//cpu_list command to view the cpu_core_id corresponding to the queue ID of the enhanced SSD whose device name is in /dev/vd* format.