Modify the TCP TIME-WAIT timeout to improve network performance - Alibaba Cloud Linux

By default, the TCP/IP TIME_WAIT state in the Linux kernel lasts for a fixed 60-second period. In certain high-load TCP scenarios, reducing this timeout can improve network performance. To enable this, Alibaba Cloud Linux 2 (kernel version 4.19.43-13.al7 and later) and Alibaba Cloud Linux 3 introduce a new kernel parameter to modify the TCP TIME_WAIT timeout to improve network performance and resource utilization.

Important

Starting from Alibaba Cloud Linux 4, this system parameter is deprecated.

Background

The TIME_WAIT state in the TCP/IP protocol is a waiting period after an application closes a communication socket. During this state, which lasts 60 seconds by default, the TCP/IP stack keeps the socket open to ensure that all data is successfully transferred between the server and the client. When an excessive number of connections accumulate in the TIME_WAIT state, it can degrade network performance. To mitigate this, Alibaba Cloud Linux provides a parameter to modify the TCP TIME_WAIT timeout. This can improve network performance in specific high-concurrency scenarios. The timeout value can be set from 1 to 600 seconds.

Usage notes

Reducing the TIME_WAIT timeout below 60 seconds contradicts the TCP/IP protocol's "quiet time" concept. This can lead to risks, such as accepting old, delayed packets as new data or rejecting new, valid packets as duplicates. Therefore, adjust this value only under the guidance of a networking expert. For more information about the quiet time concept, refer to IETF RFC 793.
Due to differences in kernel versions and parameter configurations, such as tcp_tw_reuse (see System configuration optimization) and tcp_tw_recycle (see Differences between Alibaba Cloud Linux 3 and CentOS 7), the number of connections in the TIME_WAIT state on Alibaba Cloud Linux may be higher than on other Linux distributions like CentOS 7. This is expected behavior.
Under the default Alibaba Cloud Linux configuration, the number of connections in the TIME_WAIT state typically does not exceed 5,000, and the impact on network performance is negligible.

Configuration methods

You can modify the TCP TIME_WAIT timeout in the following ways. The [$TIME_VALUE] placeholder represents the new timeout value in seconds.

Important

Modifying this parameter can cause service disruptions. Perform this operation during off-peak hours to minimize business impact.

Use the sysctl command to modify the timeout. This change takes effect immediately but resets on reboot.
```
sudo sysctl -w "net.ipv4.tcp_tw_timeout=[$TIME_VALUE]"
```
Write the value directly to the /proc/sys/net/ipv4/tcp_tw_timeout file. This change also takes effect immediately and resets on reboot.
```
sudo sh -c "echo [$TIME_VALUE] > /proc/sys/net/ipv4/tcp_tw_timeout"
```
Edit the /etc/sysctl.conf file to make the change persistent across reboots. Follow these steps:
1. Open the /etc/sysctl.conf file with a text editor such as vim.
```
sudo vim /etc/sysctl.conf
```
2. Press i to enter the insert mode. Add or modify the following line at the end of the file.
```
net.ipv4.tcp_tw_timeout = [$TIME_VALUE]
```
3. Press Esc, type :wq, and press Enter to save and exit the file.
4. Apply the changes without rebooting.
```
sudo sysctl -p
```
5. Verify that the value has been updated.
```
sysctl net.ipv4.tcp_tw_timeout
```

Note

You can run the netstat -ant | grep TIME_WAIT | wc -l command to count the number of connections currently in the TIME_WAIT state.

For example, in scenarios with a high volume of short-lived connections, such as a server running Nginx as a Layer 7 proxy, Alibaba Cloud recommends setting [$TIME_VALUE] to 5 for a 5-second TIME_WAIT timeout.