Elastic Compute Service:NIC multi-queue

Why use NIC multi-queue

A traditional single-queue network interface relies on one CPU core to process all packets. This can lead to CPU overload, increased latency, and packet loss. Modern servers have multi-core CPUs, and NIC multi-queue distributes network traffic across these cores for more efficient resource utilization.

Under the same network PPS and bandwidth conditions, tests show that using two queues instead of one improves network performance by 50% to 100%. Using four queues provides even more significant gains:

• Better utilization of multi-core CPU architecture: Distributes network traffic across multiple CPU cores for a more balanced load and higher CPU utilization.

• Higher throughput: Processing multiple packets simultaneously significantly increases network throughput, especially under high loads.

• Lower latency: Reduces congestion by distributing packets across multiple queues.

• Reduced packet loss: Mitigates packet loss caused by an overloaded single queue during periods of high traffic.

Although NIC multi-queue offers these benefits, improper configuration can decrease performance or cause other issues. For example, incorrect settings for the queue count or CPU affinity can cause unnecessary context-switching overhead. Setting the queue count too low can underutilize your hardware resources.

Typically, when an Elastic Network Interface (ENI) is attached to an instance, its queue count is automatically set to the default value for that instance type, and this setting takes effect within the operating system. If you need to manually adjust the queue count for the ENI, you should first carefully consider your specific use case and hardware conditions to determine the optimal configuration.

How NIC multi-queue works

• Queue architecture

  An Elastic Network Interface (ENI) supports multiple Combined queues. Each Combined queue is processed by an independent CPU core, enabling parallel packet processing, reducing lock contention, and fully utilizing multi-core performance.

  Receive (RX) and Transmit (TX) queues are the two types of queues used for processing packets. Each Combined queue consists of one RX queue and one TX queue:

  • RX queue: An RX queue is used to process incoming data packets from the network. When a packet arrives, the network interface distributes it to a specific RX queue based on a distribution strategy, such as round-robin or flow-based hashing.

  • TX queue: A TX queue is used to manage outgoing data packets. An application places generated packets into a TX queue, and the network interface then sends them based on factors like sequence or priority.

• IRQ affinity support

  Each queue is associated with an independent interrupt. IRQ affinity distributes interrupt handling across different CPU cores to prevent any single core from being overloaded.

  IRQ affinity is enabled by default on all public images except for Red Hat Enterprise Linux (RHEL). For more information, see Configure IRQ affinity.

Limitations

• Only some instance types support NIC multi-queue. For more information, see Instance type families. If the value in the NIC queues column is greater than 1, the instance type supports NIC multi-queue.

  • For instance types that support NIC multi-queue, the feature is enabled automatically after you attach an ENI to an instance.

  • The queue count listed for an instance type represents the maximum queues per ENI supported by that type.

  • You can call the DescribeInstanceTypes API operation to query queue-related metrics for an instance type family by specifying the InstanceTypeFamily parameter:

    • Default queue count

      The PrimaryEniQueueNumber response parameter indicates the default queue count for the primary ENI. The SecondaryEniQueueNumber parameter indicates the default queue count for a secondary ENI.

    • Maximum queues per ENI

      The MaximumQueueNumberPerEni response parameter indicates the maximum queues per ENI allowed for the instance type family.

    • Total queue quota

      The TotalEniQueueQuantity response parameter indicates the total queue quota allowed for the instance type family.

• Some older public images with kernel versions earlier than 2.6 may not support NIC multi-queue. We recommend that you use the latest public images.

View ENI queue count

Modify ENI queue count

Although an ENI's queue count is set to a default value when attached to an instance, you can manually adjust it through the console or by using the API.

• You can modify the queue count for an ENI only when it is Available, or when it is InUse and its attached instance is Stopped.

• The queue count cannot exceed the maximum queues per ENI for the instance type.

• The total queue count of all ENIs on an instance cannot exceed the total queue quota for the instance type.

After you modify the queue count for an ENI that is attached to an instance, the new setting takes effect after the instance starts.

Modify OS queue count

After you modify the queue count for an ENI in the console or by using the API, the new setting automatically takes effect within the OS. The number of queues the ENI uses within the OS matches the queue count you set.

You can also tune the number of queues that the ENI actively uses within the OS. This value can be lower than the configured queue count for the ENI.

The following example shows how to adjust the number of queues used by an ENI on an Alibaba Cloud Linux 3 instance that supports NIC multi-queue.

1. Connect to a Linux instance.

   For more information, see Log on to a Linux instance by using Workbench.

2. Run the ip address show command to view network configuration information.

   

3. Run the following command to check if the primary ENI eth0 supports NIC multi-queue.

   This example uses the primary ENI. To check a secondary ENI, replace the network interface identifier with eth1, eth2, or another value.

   ethtool -l eth0

4. Check the command output to determine if NIC multi-queue is supported:

   • If the value of "Combined" under "Pre-set maximums" is greater than 1, the ENI supports NIC multi-queue. This value indicates the maximum queue count supported by the ENI.

   • The value of "Combined" under "Current hardware settings" indicates the current queue count in use.

   In this example, the output indicates that the ENI supports a maximum of three combined (RX+TX) queues and is currently using three queues.

   

5. Run the following command to set the number of active queues for the primary ENI eth0 to 2.

   This example uses the primary ENI. To adjust a secondary ENI, replace the network interface identifier with eth1, eth2, or another value.

   sudo ethtool -L eth0 combined N

   N is the number of queues that you want the ENI to use. N must be less than or equal to the "Combined" value under "Pre-set maximums".

   In this example, set the number of queues for the primary ENI to 2:

   sudo ethtool -L eth0 combined 2

Configure IRQ affinity

When using NIC multi-queue, you typically need to configure IRQ affinity. This process assigns interrupts from different queues to specific CPU cores instead of allowing any core to handle them, which helps reduce CPU contention and improve network performance.

The following steps show how to use the ecs_mq script to automatically configure IRQ affinity for NIC multi-queue on a Red Hat Enterprise Linux 9.2 image. If your instance does not use an RHEL image, IRQ affinity is enabled by default and requires no configuration.

1. Connect to a Linux instance.

   For more information, see Log on to a Linux instance by using Workbench.

2. (Optional) Stop the irqbalance service.

   The irqbalance service dynamically adjusts IRQ affinity, which conflicts with the ecs_mq script. We recommend that you stop the irqbalance service.

   systemctl stop irqbalance.service

3. Run the following command to download the latest version of the ecs_mq automatic configuration script.

   wget https://ecs-image-tools.oss-cn-hangzhou.aliyuncs.com/ecs_mq/ecs_mq_latest.tgz

   Fixes in the latest version of ecs_mq

   The issues fixed in the latest version of ecs_mq compared to the previous version of ecs_mq are as follows:

   • Prioritizes binding interrupts to CPUs on the same Non-Uniform Memory Access (NUMA) node as the ENI's PCIe device.

   • Optimizes tuning logic for multiple network devices.

   • Adjusts interrupt binding logic based on the ratio of NIC queues to CPU cores.

   • Optimizes interrupt binding based on the position of CPU siblings.

   • Fixes increased memory access latency caused by cross-NUMA bindings in the previous version.

   • Fixes an issue on some systems where the ethtool command could not be found during ENI hot-plugging, which caused the ecs_mq_rps_rfs script to fail.

   • The new mode of the script is enabled by default. You can use the following commands to switch between the new and old modes of the ecs_mq script:

     • Switch to the old version of ecs_mq: ecs_mq_rps_rfs old

     • Switch to the new version of ecs_mq: ecs_mq_rps_rfs new

   Note

   In network performance tests, the new ecs_mq improves network performance by 5% to 30% for most PPS and BPS compared to the old ecs_mq.

4. Run the following command to decompress the ecs_mq script package.

   tar -xzf ecs_mq_latest.tgz

5. Run the following command to change the working directory.

   cd ecs_mq/

6. Run the following command to run the ecs_mq installation script.

   bash install.sh redhat 9

   Note

   Replace redhat and 9 with the name and major version number of your operating system.

7. Run the following command to start the ecs_mq service.

   systemctl start ecs_mq

   After the service starts, it automatically enables IRQ affinity.

Modifying the NIC queue count and configuring IRQ affinity are different methods for optimizing network performance. You should test different configuration combinations based on your system's actual load. Monitor performance metrics such as throughput and latency. Use this data to allocate queues to different CPU cores and set IRQ affinity to achieve optimal load balancing.