If memory usage on your Tair (Redis OSS-compatible) instance suddenly spikes to near 100%, use this guide to identify the cause and resolve it.
Common causes
A sudden memory spike has one of four causes. Use the key indicator in each row to identify which applies.
|
Cause |
Key indicator |
|
Inbound traffic and write QPS spike alongside memory usage |
|
|
Connection count spikes alongside memory usage |
|
|
Bandwidth usage reaches 100%; |
|
|
|
Large volume of new data written
A rapid influx of new data directly increases memory consumption.
Diagnose
On the Performance Monitor page, check inbound traffic and write queries per second (QPS). If both metrics follow the same trend as memory usage, the spike is due to a large volume of new data.
Resolve
Set appropriate time-to-live (TTL) values on keys to automatically expire data that is no longer needed. Alternatively, delete unnecessary keys manually.
Increase the memory capacity of the instance. For details, see Change the configurations of an instance.
If your instance is a standard instance and memory usage remains high after increasing capacity, upgrade to a cluster instance. Cluster instances distribute data across multiple shards, reducing memory pressure on each shard. For details, see Change the configurations of an instance.
Surge in new connections
Each client connection reserves memory for its input and output buffers. A sudden surge in connections causes a corresponding rise in total memory usage.
Diagnose
On the Performance Monitor page, check the number of connections. If the connection count rises sharply alongside memory usage, a connection surge is the cause.
Resolve
Check for connection leaks in your application.
Configure a connection timeout to automatically close idle connections. For details, see Specify a timeout period for client connections.
Burst traffic causing buffer backlog
When burst access generates traffic that exceeds the instance's network bandwidth, incoming and outgoing data accumulates in the input and output buffers. This buffer backlog consumes additional memory on top of normal data storage.
Diagnose
Check whether inbound and outbound traffic usage has reached 100%.
-
Run
MEMORY STATSand check whetherclients.normaloccupies an excessive amount of memory.Noteclients.normalreports the total memory consumed by the input and output buffers across all normal client connections. A high value here confirms that buffer backlog is driving the memory spike.
Resolve
Identify the cause of the traffic burst.
Increase the network bandwidth of the instance. For details, see Manually increase the bandwidth of an instance and Enable bandwidth auto scaling.
Upgrade the instance specifications to provide sufficient buffer capacity. For details, see Change the configurations of an instance.
Slow client causing output buffer backlog
When a client cannot consume responses fast enough, undelivered data accumulates in the server-side output buffer. Redis allocates memory for each queued response, which can grow large when clients fall behind.
Diagnose
In redis-cli, run MEMORY DOCTOR and check the value of big_client_buf. If big_client_buf equals 1, at least one client has an oversized output buffer consuming significant memory.
Resolve
Run CLIENT LIST and look for clients with a high omem value. The omem field reports the memory used by a client's output buffer. Check whether the identified client has performance issues preventing it from consuming responses fast enough.