MTUs - Elastic Compute Service - Alibaba Cloud Documentation Center

The maximum transmission unit (MTU) of a network connection is the size of the largest packet that can be transmitted over the connection. An MTU includes the size of IP headers and payload and excludes the size of Ethernet headers. The larger the MTU of a connection, the more data that can be passed in a single packet. This topic describes jumbo frames and how to change the MTUs of network interfaces on Elastic Compute Service (ECS) instances.

Jumbo frames

Jumbo frames are Ethernet frames that have more than 1500 bytes of payload, the limit set by the IEEE 802.3 standard. Jumbo frames increase the payload size per packet and the percentage of the packet that is not packet overhead to provide high throughput and high network performance. Alibaba Cloud supports jumbo frames that are 8,500 bytes in size and allows you to send Ethernet frames that have 8,500 bytes of payload.

All ECS instances support 1,500 MTU. Instances of the following 8th-generation instance families that are available for purchase support 1,500 MTU and 8,500 MTU: g8a, c8a, and r8a. For more information, see Overview of instance families.

The MTUs of network paths and the MTUs of peer hosts are also factors that determine whether jumbo frames can be used. Packets whose size exceeds the MTUs are fragmented or dropped by Alibaba Cloud forwarding components. The MTU of a network path is the minimum of the MTUs of each hop in the path. For more information, see Handling of packets that exceed the MTU of a network path.

Important Connectivity or performance issues may occur when jumbo frames are used in the following scenarios:

When UDP or Internet Control Messages Protocol (ICMP) jumbo frames are used to access ECS instances or hosts that are associated with Server Load Balancer (SLB) instances, fragmented packets may not be forwarded as expected by the SLB instances and then may be dropped. This causes a network connectivity issue. For more information, see the MTUs for cloud network services section of this topic.
When UDP or ICMP jumbo frames are used to establish communication in MTU mismatch scenarios, packets may be fragmented and network performance may be degraded. For more information, see the MTUs of network paths between hosts section and MTUs for cloud network services section of this topic.

Before you deploy your business on ECS instances for which jumbo frames are enabled, we recommend that you test your business on the instances to check whether connectivity or performance issues occur due to MTU mismatch and manage packet sizes to reduce performance loss.

Enable or disable jumbo frames

Use one of the following methods to enable or disable jumbo frames for an ECS instance:

Method 1: Enable or disable jumbo frames when you create an ECS instance. For more information, see Create an instance by using the wizard.
Method 2: After an ECS instance is created, enable or disable jumbo frames on the instance details page.
1. In the Network Information section on the Instance Details tab, choose > Modify Jumbo Frames Configurations.
2. In the Modify Jumbo Frames Configurations dialog box, enable or disable jumbo frames.
Method 3: Call the ModifyInstanceAttribute operation to enable or disable jumbo frames.

After you use Method 2 or Method 3 to enable or disable jumbo frames for an ECS instance, you must perform the following steps based on the operating system of the instance to make the configurations take effect:

Restart the instance.
Optional:Log on to the instance and modify the MTU settings. This operation is required only if you enable jumbo frames for a Windows instance. For more information, see the Change the MTU values for network interfaces on ECS instances section of this topic.
Note
- In Linux, Dynamic Host Configuration Protocol (DHCP) servers automatically obtain MTUs. If you restart a Linux instance after you enable jumbo frames for the instance, 8,500 MTU is automatically enabled and 1,500 MTU is automatically disabled on the instance without the need to manually modify the MTU settings.
- After you disable jumbo frames for a Windows instance and restart the instance, the standard MTU (1,500 MTU) is automatically enabled and takes effect. You do not need to log on to the instance to modify the MTU settings.

MTUs of network paths between hosts

By default, paths on Alibaba Cloud networks support 1,500 MTU. For ECS instances that do not support jumbo frames, consider only the MTUs of the associated network devices such as Virtual Private Network (VPN) gateways. For more information, see the MTUs for cloud network services section of this topic.

When an ECS instance that supports jumbo frames is used as a source host, the maximum MTU supported by the destination host varies with scenarios. The following table describes the maximum MTUs.


Destination host	Supported maximum MTU
ECS instance that has jumbo frames enabled and resides in the same VPC or a peered VPC	8500
ECS instance that has jumbo frames disabled and resides in the same VPC or a peered VPC	1500
ECS instance in a VPC that is connected by using Cloud Enterprise Network (CEN)
Host in an on-premises data center that is connected by using Connect Express
External host that can be accessed by using an elastic IP address or pip

For more information, see Overview of VPC peering connections and Manage VPC connections.

Important If packets exceed the MTU limit on a network path, Alibaba Cloud forwarding components fragment the packets or perform negotiations based on RFC standards. This helps ensure network connectivity, but does not ensure network communication performance. For paths that support a maximum MTU of 1,500 bytes, we recommend that you do not use jumbo frames for UDP or ICMP communication. For information about how forwarding components handle the packets that exceed the MTU limit on a network path, see the Handling of packets that exceed the MTU of a network path section of this topic.

MTUs for cloud network services

When you access specific network services over non-TCP protocols such as UDP and ICMP, take note of the MTU limits on the network services. Most Alibaba Cloud services do not support jumbo frames. The following table describes the maximum MTUs that are supported by specific Alibaba Cloud network services.


Network service	Supported maximum MTU	Description
NAT Gateway	1500	None.
SLB, including Application Load Balancer (ALB), Network Load Balancer (NLB), and Classic Load Balancer (CLB)		ALB instances and CLB instances cannot forward fragmented packets. Set the size limit of each packet that is sent from ECS instances to 1,500 bytes for UDP or ICMP communication.
Other Alibaba Cloud services, such as Object Storage Service (OSS) and ApsaraDB RDS		Traffic goes through SLB services. Take note of the limits on the SLB services.
Global Accelerator		None.
Cloud Firewall		None.
PrivateLink		None.
VPN Gateway	1399	For more information, see Set MTU values.

Important If packets exceed the MTU limit on a cloud service, Alibaba Cloud forwarding components fragment the packets or perform negotiations based on RFC standards. This helps ensure network connectivity, but does not ensure network communication performance. For cloud services that support a maximum MTU of 1,500 or 1,399 bytes, we recommend that you do not use jumbo frames for UDP or ICMP communication. For information about how forwarding components handle the packets that exceed the MTU limit on a cloud service, see the Handling of packets that exceed the MTU of a network path section of this topic.

Handling of packets that exceed the MTU of a network path

To ensure network connectivity, Alibaba Cloud forwarding components use one of the following features to handle the packets that exceed the MTU of a network path based on RFC standards:

TCP maximum segment size (MSS) for TCP traffic

TCP MSS limits the size of TCP segments, excluding TCP headers and IP headers, to prevent packet fragmentation. To establish a TCP connection between a source host and a destination host, both hosts advertise an MSS value. The MSS value is calculated based on the following formula: MSS = MTU - (IP header size + TCP header size). To ensure that packets can be sent and received as expected along a network path, the MSS plus TCP headers (20 bytes) plus IP headers (20 bytes per IPv4 header or 40 bytes per IPv6 header) must be less than or equal to the MTU of the network path.

If the MSS is not appropriate for the MTU of the network path, Alibaba Cloud forwarding components perform MSS clamping to lower the MSS value advertised by the source host in a Synchronize Sequence Number (SYN) packet or the MSS value advertised by the destination host in a SYN-ACK packet.

PMTUD for non-TCP traffic (UDP traffic or ICMP traffic)

For non-TCP packets, Alibaba Cloud forwarding components use the Path MTU Discovery (PMTUD) mechanism instead of TCP MSS to detect and negotiate MTUs.

When packets exceed the MTU of a network path, Alibaba Cloud forwarding components fragment the packets that do not include the Don't-Fragment flag based on the MTU, and drop the packets in which the Don't-Fragment flag is set to 1. After a packet is dropped, the forwarding components perform operations based on the following scenarios:

If the Don't-Fragment flag is set to 1 in the IPv4 header of the packet, the forwarding components send an ICMP Fragmentation-Needed packet that indicates the MTU of the network path to the sender of the packet.
If the Don't-Fragment flag is set to 1 in the IPv6 header of the packet, the forwarding components send an ICMPv6 Packet-Too-Long packet that indicates the MTU of the network path to the sender of the packet.

Important

PMTUD causes packet loss and helps ensure only network connectivity. Resources in the kernel and underlying forwarding components of ECS instances are consumed to construct and respond to PMTU packets and handle fragmented packets. PMTUD significantly affects performance. To improve performance, we recommend that you call the ModifyInstanceAttribute operation to disable jumbo frames for ECS instances and then restart the instances for the change to take effect.
To allow an ECS instance to respond to PMTUD, make sure that ICMP inbound rules are configured in the security groups of the instance to receive ICMP packets from different forwarding components. You must also make sure that the operating system of the instance supports PMTUD and that PMTU response is enabled for the operating system. For example, set the required options in /proc/sys/net/ipv4/ip_no_pmtu_disc to 0 to enable PMTU response for Linux.

Change the MTU values for network interfaces on ECS instances

In Alibaba Cloud, specific scenarios or network devices, such as VPC gateways and traffic mirroring, have strict requirements on the sizes of packets that are sent from ECS instances. To ensure network connectivity or improve user experience, you must reduce the packet size. You can perform the following operations to change the MTU values for network interfaces on ECS instances to meet the packet size requirements of the preceding scenarios or network devices.

Warning When you change the MTU values of network interfaces on an ECS instance, make sure that the specified MTU values do not exceed the MTU limit at the instance level. If the specified MTU values exceed the MTU limit at the instance level, the instance may lose network connectivity. The MTU at the instance level is 8,500 bytes when jumbo frames are enabled, and 1,500 bytes when jumbo frames are disabled or when the instance does not support jumbo frames.

In Linux, DHCP servers automatically obtain MTUs except in tunneling scenarios, such as the scenarios in which VPN gateways are used. This way, you do not need to change the MTU values of network interfaces.
In Windows, DHCP servers do not automatically obtain MTUs. When you enable or disable jumbo frames for Windows instances, perform the operations to change the MTU values of network interfaces on the instances.

The following sections provide examples how to change the MTU values of network interfaces to 1,399 bytes on ECS instances that run different operating systems.

Windows

In this example, an ECS instance that runs Windows Server 2019 is used. To change the MTU value of a network interface on the instance, perform the following steps:

Connect to the Windows instance.
For more information, see Connect to a Windows instance by using a password or key.
Start PowerShell.
Run the following command to view the network interfaces of the instance:
```
Get-NetAdapter
```
The following figure shows a sample command output that indicates that a network interface is named Ethernet.
Run the following command to view the MTU value of the Ethernet network interface:
```
Get-NetAdapterAdvancedProperty -Name "Ethernet"
```
The following figure shows a sample command output that indicates that the MTU value of the Ethernet network interface is 1,514.
Note In Windows, the methods that are used to calculate MTUs vary based on drivers. In most cases, drivers include the Ethernet header that is 14 bytes in size in MTU calculation to obtain the default MTU of 1,514 bytes.
Run the following command to change the MTU value of a network interface:
```
Get-NetAdapterAdvancedProperty -Name <"Network interface name"> -RegistryKeyword "*JumboPacket" -RegistryValue <New MTU value>
```
For example, run the following command to set the MTU value of the Ethernet network interface to 1,413. This value includes the 14 bytes of the Ethernet header.
```
Set-NetAdapterAdvancedProperty -Name "Ethernet" -RegistryKeyword "*JumboPacket" -RegistryValue 1413
```
After you change the MTU value of the network interface, run the Get-NetAdapterAdvancedProperty -Name "Ethernet" -RegistryKeyword "*JumboPacket" command to check whether the new MTU value takes effect.
The following figure shows a sample command output that indicates that the MTU value of the Ethernet network interface is changed to 1,413.

Ubuntu

In this example, an ECS instance that runs Ubuntu16.04 is used. To change the MTU value of a network interface on the instance, perform the following steps:

Connect to the instance.
For more information, see Connect to a Linux instance by using a password or key.
Run the following command to view the network interfaces of the instance and the MTU values of the network interfaces:
```
ifconfig
```
The following figure shows a sample command output that indicates that the MTU value of the network interface named eth0 is 1,500.
Change the MTU value of the eth0 network interface.
For example, run the following commands to set the MTU value of the eth0 network interface to 1,399:
```
mtu=1399
ifconfig eth0 mtu ${mtu}
echo "/sbin/ifconfig eth0 mtu ${mtu}" > /etc/network/if-up.d/setmtu && chmod a+x /etc/network/if-up.d/setmtu
```
After you change the MTU value of the network interface, run the ifconfig command to check whether the new MTU value takes effect.
The following figure shows a sample command output that indicates that the MTU value of the eth0 network interface is changed to 1,399.

CentOS

In this example, an ECS instance that runs CentOS 7.8 is used. To change the MTU value of a network interface on the instance, perform the following steps:

Connect to the instance.
For more information, see Connect to a Linux instance by using a password or key.
Run the following command to view the network interfaces of the instance and the MTU values of the network interfaces:
```
ifconfig
```
The following figure shows a sample command output that indicates that the MTU value of the network interface named eth0 is 1,500.
Change the MTU value of the eth0 network interface.
For example, run the following commands to set the MTU value of the eth0 network interface to 1,399:
```
mtu=1399
echo "${mtu}" > /sys/class/net/eth0/mtu
echo "echo \""${mtu}"\" > /sys/class/net/eth0/mtu">> /etc/rc.d/rc.local && chmod +x /etc/rc.d/rc.local
```
After you change the MTU value of the network interface, run the ifconfig command to check whether the new MTU value takes effect.
The following figure shows a sample command output that indicates that the MTU value of the eth0 network interface is changed to 1,399.

Common issues

Problem description: When you use jumbo frames on an ECS instance for UDP or ICMP traffic, the performance of the instance drops significantly.
Solution: Check whether packets are fragmented and disable jumbo frames for the instance based on your business requirements. For more information, see the Enable or disable jumbo frames section of this topic.
Problem description: When you access cloud services (such as OSS and AparaDB for RDS) over UDP or ICMP from an ECS instance for which jumbo frames are enabled, network connections cannot be established to the services.
Cause: Traffic from the ECS instance flows through SLB instances that cannot forward packet fragments. As a result, a network connection failure may occur.
Solution: To ensure that packets sent by the ECS instance to the cloud services are not fragmented, set the size limit of each packet to 1,500 bytes.
Problem description: An ECS instance cannot receive PMTU messages (ICMP error packets).
Cause: PMTU messages are dropped by the security groups of the ECS instance or throttled.
Solution:
1. Check whether the security groups of the instance allows ICMP traffic. For more information, see View security group rules.
2. Check whether the traffic rate of the instance reaches the throttling limit.