Elastic Compute Service:NIC multi-queue

Benefits of the NIC multi-queue feature

A traditional single-queue NIC uses a single CPU core to process all packets, which easily leads to CPU overload, increased latency, and packet loss. Modern servers are equipped with multi-core CPUs. A multi-queue NIC can distribute network traffic to different CPU cores. This improves the utilization of the multi-core computing resources.

Test results indicate that under identical packet forwarding rate and network bandwidth conditions, two queues outperform a single queue by 50% to 100% and performance improvements are significantly higher when four queues are used. The NIC multi-queue feature provides the following benefits:

• Improve the utilization of the multi-core CPU architecture. A multi-queue NIC can distribute network traffic to multiple CPU cores for processing. This can better load-balance CPU cores and improve CPU utilization.

• Improve throughput. A multi-queue NIC allows multiple packets to be simultaneously processed, which can significantly improve the throughput of the network, especially under high loads.

• Reduce latency. Packets can be distributed to different queues for processing. This reduces congestion in a single queue and reduces latency.

• Reduce packet loss. In high-traffic scenarios, the NIC multi-queue feature can reduce packet loss caused by the overload of a single queue.

In most cases, the NIC multi-queue feature provides the preceding benefits. However, improper configuration of the feature may cause performance degradation or other issues. For example, improper configuration of the number of queues supported by a NIC and CPU affinity may cause unnecessary context switching overheads. If you set the number of queues supported by a NIC to a small value, you cannot make full use of hardware resources.

In most cases, after you bind an elastic network interface (ENI) to an Elastic Compute Service (ECS) instance, the number of queues supported by the ENI is automatically set to the default number of queues supported by the instance type per ENI and takes effect in the operating system. If you want to manually change the number of queues supported by an ENI, consider how to properly configure the multi-queue parameters of the ENI based on your business scenario and hardware conditions.

Core mechanisms of the NIC multi-queue feature

• Queue division

  An ENI supports multiple Combined queues. Each Combined queue is processed by an independent CPU core. This way, packets in different queues can be processed in parallel. This reduces lock contention and fully utilizes multi-core performance.

  Rx queues and Tx queues are two different types of queues used by ENIs to process packets. Each Combined queue consists of an Rx queue and a Tx queue.

  • Rx queue: is used to process packets received from the network. When packets arrive at an ENI, the packets are distributed to different Rx queues based on specific rules, such as polling and flow-based distribution.

  • Tx queue: manages the packets to be sent. Output requests generated by applications are first placed into the corresponding Tx queues and then sent by an ENI based on various factors such as order or priority.

• Support for IRQ Affinity

  Each queue is associated with an independent interrupt. Interrupt Request (IRQ) Affinity is used to distribute interrupts for different queues to specific CPU cores to prevent the overload of a single CPU core.

  By default, IRQ Affinity is enabled in all images without the need for additional configurations, except Red Hat Enterprise Linux images. For more information, see the Configure IRQ Affinity section of this topic.

Constraints and limits

• For information about the instance types that support the NIC multi-queue feature, see Overview of instance families. If the value in the NIC queues column for an instance type is greater than 1, the instance type supports the NIC multi-queue feature.

  • For an instance type that supports the NIC multi-queue feature, the feature automatically takes effect on an instance of the instance type after you bind an ENI to the instance.

  • The value in the NIC queues column for an instance type in the instance type tables indicates the maximum number of queues allowed per ENI for the instance type.

  • You can call the DescribeInstanceTypes operation to query the metrics about the number of NIC queues for the instance family specified by the InstanceTypeFamily parameter.

    • The default number of queues supported by an ENI

      The value of the PrimaryEniQueueNumber parameter in the response indicates the default number of queues supported by the primary ENI. The value of the SecondaryEniQueueNumber parameter indicates the default number of queues supported by a secondary ENI.

    • The maximum number of queues supported by an ENI

      The value of the MaximumQueueNumberPerEni parameter in the response indicates the maximum number of queues allowed per ENI for the specified instance family.

    • The queue quota

      The value of the TotalEniQueueQuantity parameter in the response indicates the queue quota of the specified instance family.

• Specific early-version public images that contain kernel versions earlier than 2.6 may not support the NIC multi-queue feature. We recommend that you use the latest public images.

View the number of queues supported by an ENI

Change the number of queues supported by an ENI

In most cases, after you bind an ENI to an ECS instance, the number of queues supported by the ENI is automatically set to the default number of queues supported by an ENI for the instance type and takes effect in the operating system. If you want to manually change the number of queues supported by an ENI, you can use the ECS console or call an API operation. Take note of the following items:

• You can change the number of queues supported by an ENI only if the ENI is in the Available state or if the ENI is in the InUse state and the ECS instance to which the ENI is bound is in the Stopped state.

• The number of queues supported by an ENI that is bound to an ECS instance cannot exceed the maximum number of queues allowed per ENI for the instance type.

• The total number of queues for all ENIs on an ECS instance cannot exceed the queue quota allowed for the instance type.

If you change the number of queues supported by an ENI that is bound to an ECS instance, the new number of queues takes effect after the instance starts.

Change the number of queues that can be used by an ENI in the operating system

After you change the number of queues supported by an ENI in the ECS console or by calling an API operation, the change automatically takes effect in the operating system.

You can change the number of queues that can be used by an ENI in the operating system to a number that is smaller than the number of queues supported by the ENI.

In the following example, an ECS instance that runs Alibaba Cloud Linux 3 is used. Alibaba Cloud Linux 3 supports the NIC multi-queue feature.

1. Connect to a Linux ECS instance.

   For more information, see Use Workbench to connect to a Linux instance over SSH.

2. Run the ip address show command to view the network configurations.

   

3. Run the following command to check whether the NIC multi-queue feature is enabled on the primary ENI eth0.

   In this example, the primary ENI is used. If you use a secondary ENI, replace the ENI name with the actual ENI name, such as eth1 or eth2.

   ethtool -l eth0

4. View the command output to check whether the NIC multi-queue feature is enabled.

   • If the value of the Combined field in the Pre-set maximums section is greater than 1, the ENI supports the multi-queue feature. The value of the Combined field in the Pre-set maximums section indicates the maximum number of queues supported by the ENI.

   • The value of the Combined field in the Current hardware settings section indicates the number of queues that can be used by the ENI.

   In this example, the command output indicates that the ENI supports up to three Combined queues and uses three Combined queues.

   

5. Run the following command to change the number of queues that can be used by the primary ENI eth0 to 2.

   In this example, the primary ENI is used. If you use a secondary ENI, replace the ENI name with the actual ENI name, such as eth1 or eth2.

   sudo ethtool -L eth0 combined N

   N specifies the number of queues that you want the ENI to use. N must be less than or equal to the value of the Combined field in the Pre-set maximums section.

   In this example, the primary ENI is configured to use two queues.

   sudo ethtool -L eth0 combined 2

Configure IRQ Affinity

When you use the NIC multi-queue feature, you must configure IRQ Affinity to assign interrupts for different queues to specific CPUs, instead of allowing the interrupts to be assigned to arbitrary CPUs. This helps reduce contention among CPUs and improve network performance.

This section describes how to use the ecs_mq script to configure IRQ Affinity for a Linux ECS instance that uses a Red Hat 9.2 image. If the image is not a Red Hat Enterprise Linux image, IRQ Affinity is enabled by default. You do not need to configure IRQ Affinity.

1. Connect to a Linux ECS instance.

   For more information, see Use Workbench to connect to a Linux instance over SSH.

2. (Optional) Disable the irqbalance service.

   The irqbalance service dynamically modifies IRQ Affinity configurations. The ecs_mq script conflicts with the irqbalance service. We recommend that you disable the irqbalance service.

   systemctl stop irqbalance.service

3. Run the following command to download the package that contains the new version of the ecs_mq script:

   wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/ecs_mq/ecs_mq_2.0.4.tgz

   Benefits of the new version of the ecs_mq script

   Compared with the old version of the ecs_mq script, the new version provides the following advantages:

   • Preferentially binds interrupts for an ENI to CPUs on the Non-Uniform Memory Access (NUMA) node with which the Peripheral Component Interconnect Express (PCIe) interface of the ENI is associated.

   • Optimizes the logic for tuning multiple network devices.

   • Binds interrupts for ENIs of different specifications based on the ratio of the number of queues to the number of CPUs.

   • Optimizes the mechanism of binding interrupts based on the positions of CPU siblings.

   • Resolves high latency issues that may occur during memory access across NUMA nodes.

   • Uses the new version of the ecs_mq script by default and provides the commands that can be used to switch between the old and new versions.

     • The ecs_mq_rps_rfs old command is used to switch to the old version of the ecs_mq script.

     • The ecs_mq_rps_rfs new command is used to switch to the new version of the ecs_mq script.

   Note

   In network performance tests, the new version of the ecs_mq script outperforms the old version by 5% to 30% in most PPS and BPS metrics.

4. Run the following command to extract the ecs_mq script:

   tar -xzf ecs_mq_2.0.tgz

5. Run the following command to change the working path:

   cd ecs_mq/

6. Run the following command to install the environment that is required to run the ecs_mq script:

   bash install.sh redhat 9

   Note

   Replace redhat and 9 with the actual operating system name and major version number of the operating system.

7. Run the following command to start the ecs_mq script:

   systemctl start ecs_mq

   After the script is started, IRQ Affinity is automatically enabled.

You can change the number of queues on ENIs, configure IRQ Affinity, or perform both preceding operations to optimize network performance. To ensure load balancing, assign an appropriate number of queues to each CPU core and configure IRQ Affinity based on the actual loads and performance data of the system, such as throughput and latency. To obtain the system performance data, you can test various queue and IRQ Affinity configurations.