Elastic Compute Service:Test instance network performance

Packet forwarding rate lower than 6,000,000 pps

1. Connect to the test machine and auxiliary test machines. See Log on to a Linux instance using Workbench.

2. On both machines, download the Netperf installation package:

   wget https://benchmark-packages.oss-cn-qingdao.aliyuncs.com/netperf-2.7.0.tar.gz

3. On both machines, install Netperf and sar.

   1. Decompress the Netperf installation package:

      sudo yum install -y gcc autoconf automake libtool sysstat
      tar -zxvf netperf-2.7.0.tar.gz

   2. Query the version number of GNU Compiler Collection (GCC).

      gcc -v 2>&1

   3. If GCC is later than V10 on either machine, perform the following operations to ensure that GCC compiles correctly. Otherwise, skip this step.

      1. Open the nettest_omni.c file.

         cd netperf
         vim src/nettest_omni.c

      2. Press i to enter Insert mode and delete the following declared variables from nettest_omni.c:

         /* different options for the sockets        */
         
         int
           loc_nodelay,    /* don't/do use NODELAY  locally    */
           rem_nodelay,    /* don't/do use NODELAY remotely  */
           loc_sndavoid,    /* avoid send copies locally    */
           loc_rcvavoid,    /* avoid recv copies locally    */
           rem_sndavoid,    /* avoid send copies remotely    */
           rem_rcvavoid;   /* avoid recv_copies remotely    */

      3. Press Esc, enter :wq, and press Enter to save and close the file.

   4. Install Netperf and sar.

      cd netperf
      sudo ./configure
      sudo make && sudo make install

4. On the test machine, start 64 netserver services:

   #!/bin/bash
   for j in `seq 64`; do
       netserver -p $[16000+j] > server_$[16000+j].netperf > /dev/null 2>&1  &
   done

5. Query the private IP address of the test machine:

   ifconfig || ip addr

   

6. On the auxiliary test machine, send traffic to the test machine:

   #!/bin/bash
   server_ip=<Private IP address of the test machine>
   for j in `seq 64`; do
       port=$[16000+j]
       netperf -H ${server_ip} -l ${run_time:-300} -t UDP_STREAM  -p $port  -- -m 1 -D > /dev/null 2>&1 &
   done

   Replace <Private IP address of the test machine> with the actual private IP address obtained in the previous step.

   

7. On the test machine, test the packet forwarding rate:

   sar -n DEV 1

   Check the rxpck/s column. The rxpck/s value shows packets received per second. In this example, the value is approximately 940,000.

   

Packet forwarding rate higher than 6,000,000 pps and lower than 20,000,000 pps

1. Connect to the test machine and auxiliary test machines. See Log on to a Linux instance using Workbench.

2. On all four machines (three auxiliary + one test), install sockperf:

   sudo yum install -y autoconf automake libtool gcc-c++
   cd /opt
   sudo wget https://github.com/Mellanox/sockperf/archive/refs/tags/3.8.tar.gz
   sudo tar -zxf 3.8.tar.gz
   cd sockperf-3.8/
   sudo ./autogen.sh
   sudo ./configure
   sudo make -j `cat /proc/cpuinfo| grep process | wc -l`
   sudo make install

3. Query the private IP address of the test machine:

   ifconfig || ip addr

   

4. On each auxiliary test machine, send traffic to the test machine:

   server_ip="<Private IP address of the test machine>"
   threads=64
   msg_size=14
   run_time=60
   
   basePort=6666
   for((i=0;i<$threads;++i));do
       nohup sockperf tp -i $server_ip --pps max -m ${msg_size} -t ${run_time} --port $[${basePort}+${i}] 2>&1 &
   done

   Replace <Private IP address of the test machine> with the actual value. run_time specifies the traffic duration.

5. On all four machines, test the packet forwarding rate:

   sar -n DEV 1

   Check the rxpck/s column on the test machine. The rxpck/s value shows packets received per second. In this example, the value is approximately 12,000,000 pps.

Packet forwarding rate higher than 20,000,000 pps

1. Connect to the test machine and auxiliary test machines. See Log on to a Linux instance using Workbench.

2. On all four machines (three auxiliary + one test), install sockperf:

   sudo yum install -y autoconf automake libtool gcc-c++
   cd /opt
   sudo wget https://github.com/Mellanox/sockperf/archive/refs/tags/3.8.tar.gz
   sudo tar -zxf 3.8.tar.gz
   cd sockperf-3.8/
   sudo ./autogen.sh
   sudo ./configure
   sudo make -j `cat /proc/cpuinfo| grep process | wc -l`
   sudo make install

3. On the test machine, bind interrupt requests (IRQs) to CPU cores.

   Binding IRQs to specific CPU cores reduces context switching and improves packet processing efficiency at high PPS.

   a=$(cat /proc/interrupts | grep virtio2-input | awk -F ':' '{print $1}')
   cpu=0
   for irq in $a; do
       echo $cpu >/proc/irq/$irq/smp_affinity_list
       let cpu+=2
   done

4. Query the private IP address of the test machine:

   ifconfig || ip addr

   

5. On each auxiliary test machine, send traffic to the test machine:

   server_ip="<Private IP address of the test machine>"
   threads=64
   msg_size=14
   run_time=60
   
   basePort=6666
   for((i=0;i<$threads;++i));do
       nohup sockperf tp -i $server_ip --pps max -m ${msg_size} -t ${run_time} --port $[${basePort}+${i}] 2>&1 &
   done

   Replace <Private IP address of the test machine> with the actual value. run_time specifies the traffic duration.

6. On all four machines, test the packet forwarding rate:

   sar -n DEV 1

   Check the rxpck/s column on the test machine. The rxpck/s value shows packets received per second. In this example, the value is approximately 20,000,000 pps.

Linux instance

1. Connect to the test machine and the auxiliary test machine. See Log on to a Linux instance using Workbench.

2. On both machines, download the Netperf installation package:

   wget https://benchmark-packages.oss-cn-qingdao.aliyuncs.com/netperf-2.7.0.tar.gz

3. On both machines, install Netperf and sar.

   1. Decompress the Netperf installation package.

      sudo yum install -y gcc autoconf automake libtool sysstat
      tar -zxvf netperf-2.7.0.tar.gz

   2. Query the version number of GCC.

      gcc -v 2>&1

   3. If GCC is later than V10 on either machine, perform the following operations to ensure that GCC compiles correctly. Otherwise, skip this step.

      1. Open the nettest_omni.c file.

         cd netperf
         vim src/nettest_omni.c

      2. Press i to enter Insert mode and delete the following declared variables from nettest_omni.c:

         /* different options for the sockets        */
         
         int
           loc_nodelay,    /* don't/do use NODELAY  locally    */
           rem_nodelay,    /* don't/do use NODELAY remotely  */
           loc_sndavoid,    /* avoid send copies locally    */
           loc_rcvavoid,    /* avoid recv copies locally    */
           rem_sndavoid,    /* avoid send copies remotely    */
           rem_rcvavoid;   /* avoid recv_copies remotely    */

      3. Press Esc, enter :wq, and press Enter to save and close the file.

   4. Install Netperf and sar.

      cd netperf
      sudo ./configure
      sudo make && sudo make install

4. On the test machine, start 64 netserver services:

   #!/bin/bash
   for j in `seq 64`; do
       netserver -p $[16000+j] > server_$[16000+j].netperf 2>&1 &
   done

5. Query the private IP address of the test machine:

   ifconfig || ip addr

   

6. On the auxiliary test machine, send traffic to the test machine:

   #!/bin/bash
   server_ip=<Private IP address of the test machine>
   for j in `seq 64`; do
       port=$[16000+j]
       netperf -H ${server_ip} -l ${run_time:-300} -t TCP_STREAM -p $port -- -D > /dev/null 2>&1 & 
   done

   Replace <Private IP address of the test machine> with the actual private IP address obtained in the previous step.

   

7. On the test machine, test the network bandwidth:

   sar -n DEV 1

   Check the rxkB/s column. The rxkB/s value shows received packet size in KB/s. Multiply by 8 to convert to Kbit/s for the network bandwidth value.

   

Windows instance

Linux instance

1. Connect to the test machine and the auxiliary test machine. See Log on to a Linux instance using Workbench.

2. On both machines, install sockperf:

   sudo yum install -y autoconf automake libtool gcc-c++
   cd /opt
   sudo wget https://github.com/Mellanox/sockperf/archive/refs/tags/3.8.tar.gz
   sudo tar -zxf 3.8.tar.gz
   cd sockperf-3.8/
   sudo ./autogen.sh
   sudo ./configure
   sudo make -j `cat /proc/cpuinfo| grep process | wc -l`
   sudo make install

3. Query the private IP address of the test machine:

   ifconfig || ip addr

   

4. On the test machine, start sockperf:

   sockperf sr --tcp --daemonize

5. On the auxiliary test machine, send traffic to the test machine:

   sockperf pp -i <Private IP address of the test machine> --tcp -t 30 -m 14 --full-log=result.json

   Replace <Private IP address of the test machine> with the actual value from the previous step.

6. View the network latency results.

   In the returned data:

   • avg-latency: average latency in microseconds.

   • percentile 99.000: 99th-percentile latency in microseconds.

   

Windows instance