Diagnose and resolve the Cannot assign requested address error when connecting to Tair (Redis OSS-compatible) instances over short-lived connections.
Quick solution
Use persistent connections instead of short-lived connections. For PHP applications using phpredis, replace connect() with pconnect():
// Replace this:
$redis->connect($host, $port);
// With this:
$redis->pconnect($host, $port, 0, null, 0, 0, ['auth' => [$password]]);
Problem overview
|
Aspect |
Details |
|
Error message |
|
|
Affected scenarios |
High concurrency with short-lived connections |
|
Root cause |
TCP port exhaustion due to TIME_WAIT connections |
|
Recommended solution |
Use persistent connections or connection pooling |
Prerequisites
Ensure the following:
-
Access to the client ECS instance running your application
-
Root or sudo privileges for kernel parameter modifications
-
Knowledge of your application's Redis client library
Symptoms
This error occurs when:
-
Your application creates a new connection per request
-
Traffic concurrency is high
-
You see many TCP connections in TIME_WAIT state on the client machine
Check TIME_WAIT connections:
ss -tan state time-wait | wc -l
If the count approaches your available port range (28,232 ports by default), port exhaustion is occurring.
Cause
With short-lived connections, each request creates a new TCP connection. After closing, the socket enters TIME_WAIT for approximately 60 seconds before the port becomes available again.
Under high concurrency, connections open and close faster than ports recycle, exhausting available ports and triggering the Cannot assign requested address error.
Solutions
Solution 1: Use persistent connections (Recommended)
Persistent connections or connection pooling reuse TCP connections across requests, eliminating port exhaustion.
Benefits:
-
Eliminates TIME_WAIT accumulation
-
Reduces connection latency
-
Lowers CPU overhead from TCP handshakes
PHP with phpredis
<?php
// Short-lived connection (causes the error)
$redis = new Redis();
$redis->connect($host, $port);
$redis->auth($password);
// Persistent connection (recommended)
$redis = new Redis();
$redis->pconnect($host, $port, 0, null, 0, 0, ['auth' => [$password]]);
Parameters for pconnect:
|
Parameter |
Description |
Recommended value |
|
host |
Tair instance endpoint |
Your instance endpoint |
|
port |
Tair instance port |
6379 (default) |
|
timeout |
Connection timeout in seconds |
0 (no timeout) |
|
persistent_id |
Persistent connection identifier |
null |
|
retry_interval |
Retry interval in milliseconds |
0 |
|
read_timeout |
Read timeout in seconds |
0 |
|
auth |
Authentication options |
|
For phpredis 5.3.0 or later, pass the password in the auth options array to prevent NOAUTH errors during reconnection.
Python with redis-py
import redis
# Connection pool (recommended)
pool = redis.ConnectionPool(
host='your-tair-endpoint',
port=6379,
password='your-password',
max_connections=10,
decode_responses=True
)
# Reuse the pool across requests
r = redis.Redis(connection_pool=pool)
r.set('key', 'value')
Node.js with ioredis
const Redis = require('ioredis');
// Single persistent connection (recommended)
const redis = new Redis({
host: 'your-tair-endpoint',
port: 6379,
password: 'your-password',
lazyConnect: true,
keepAlive: 10000
});
// For multiple connections, use a cluster or pool
Solution 2: Adjust TCP kernel parameters
If modifying application code is not feasible, adjust the tcp_max_tw_buckets kernel parameter to limit TIME_WAIT connections. This provides quick HA but is less effective than persistent connections.
Steps:
-
Log on to the ECS instance where your client application runs.
-
Check current TCP settings:
sysctl net.ipv4.tcp_max_tw_buckets net.ipv4.ip_local_port_rangeExample output:
net.ipv4.tcp_max_tw_buckets = 262144 net.ipv4.ip_local_port_range = 32768 60999 -
Set
tcp_max_tw_bucketsto a value smaller than your port range start value:sysctl -w net.ipv4.tcp_max_tw_buckets=10000 -
To persist across reboots, add to
/etc/sysctl.conf:net.ipv4.tcp_max_tw_buckets = 10000 -
Apply the changes:
sysctl -p
Limitations:
-
If the server is in LAST_ACK state when retransmitting packets, new connections using the same 5-tuple may fail
-
This is a workaround, not a complete solution
-
Persistent connections remain the recommended approach
Choose the right solution
|
Scenario |
Recommended solution |
|
New application development |
Persistent connections |
|
Existing code, easy to modify |
Persistent connections |
|
Complex legacy codebase |
TCP parameter adjustment |
|
Containerized applications |
Persistent connections |
|
Temporary quick fix needed |
TCP parameter adjustment |
Best practices
Deprecated parameters
tcp_tw_recycle was removed in Linux kernel 4.12. Avoid tcp_tw_recycle and tcp_tw_reuse, especially in environments with:
-
Network Address Translation (NAT)
-
Linux Virtual Server (LVS)
-
Load balancers
-
Kubernetes or container orchestration platforms
Monitoring recommendations
Monitor these metrics to prevent recurrence:
-
TIME_WAIT connection count on client machines
-
Redis connected clients (
INFO clients) -
Connection errors in application logs
-
Port utilization on client machines