Real-time search for Redis large keys and hotkeys - ApsaraDB for Redis

ApsaraDB for Redis integrates the key analysis feature of Database Autonomy Service (DAS) to provide the real-time key statistics feature. This feature provides real-time and historical statistics about large keys and hotkeys, including the number of elements in large keys and the access frequency of hotkeys. When the memory usage or CPU utilization of an ApsaraDB for Redis instance increases, you can use the real-time key statistics feature to quickly identify large keys and hotkeys to resolve performance issues in the instance.

Prerequisites

The instance is an ApsaraDB for Redis Community Edition instance that runs Redis 5.0 or later or an ApsaraDB for Redis Enhanced Edition (Tair) DRAM-based or persistent memory-optimized instance.

Statistical ranking mechanisms for large keys and hotkeys

Key type	Statistical ranking mechanism
Large keys	Supported data types Native Redis data structures: String, List, Hash, Set, Zset, and Stream. Self-developed Tair data structures: TairString, TairHash, TairGIS, TairBloom, TairDoc, TairCpc, TairZset, TairRoaring, TairTS, and TairSearch. Large keys (keys with a large number of elements) Statistical threshold: 2,000 (string length or number of elements). Number of large keys per master node: 3. If the instance uses the cluster or read/write splitting architecture, the system displays the three largest keys of each data type in each data shard or read replica. Statistical rules Keys are ranked based on the string length (for the String type) or number of elements (for other data types). If keys differ in size, the system selects the top three keys that meet or exceed the defined threshold. If multiple keys have the same size, the key that is scanned first during the statistical scan is ranked higher. Large keys (keys that occupy a large amount of memory) Note Only ApsaraDB for Redis Enhanced Edition (Tair) DRAM-based instances of version 6.0 or later are supported. Statistical threshold: A single element is larger than 50 KB or an entire key is larger than 500 MB. Number of large keys per master node: 3. If the instance uses the cluster or read/write splitting architecture, the system displays the three largest keys of each data type in each data shard or read replica. Statistical rules Keys are ranked based on the amount of memory that they consume. The MEMORY USAGE command is used to determine the memory usage of each key. If keys differ in memory usage, the system selects the top three keys that meet or exceed the defined threshold. If multiple keys have the same memory usage, they are sorted in lexicographical order. Note Keys that have not been accessed for a long period of time may not be included in the statistical ranking. You can use the offline key analysis feature to obtain information about large keys, such as the memory usage of keys, the distribution of keys, the memory usage and distribution of elements in keys, and the distribution of time-to-live (TTL) values across keys. For more information, see Use the offline key analysis feature.
Hotkeys	ApsaraDB for Redis uses efficient sorting and statistical algorithms based on the Least Recently Used (LRU) key eviction policy to identify hotkeys in an instance. Note For Redis Community Edition instances of version 7.0.18, 6.0.2.9, or 5.5.2.9 or Tair DRAM-based instances of version 5.0.50 and later, the following enhancements to hotkeys are made: Statistics about up to 50 hotkeys can be collected at the same time. In the previous versions of Redis Community Edition, only statistics about up to 20 hotkeys can be collected at the same time. A key is considered a hotkey if its queries per second (QPS) exceeds 5,000. In earlier versions, a key is considered a hotkey if its QPS exceeds 3,000. The exact QPS of a hotkey can be displayed. In earlier versions, the approximate QPS range can be displayed only if the QPS of a hotkey is less than 6,000.

Procedure

Log on to the ApsaraDB for Redis console and go to the Instances page. In the top navigation bar, select the region in which the instance that you want to manage resides. Then, find the instance and click the instance ID.
In the left-side navigation pane, choose CloudDBA > Real-time Key Statistics.
Click the Real-time tab to view real-time data or click the History tab to view historical data based on your business requirements.
Figure 1. Key analysis
Note
- If the instance is a cluster instance or a read/write splitting instance, you can select a node from the Current Node drop-down list.
- On the History tab, you can query the historical large keys and hotkeys within the previous four days. The time range to query cannot exceed 3 hours. For example, if the current time is 16:00 on July 15, 2021, you can query the large keys and hotkeys from 16:00 on July 11 to 16:00 on July 15. If you set the beginning of the time range to query to 16:00 on July 11, the end of the time range to query cannot be later than 19:00 on July 11.

Related API operations

DAS API operation	Description
DescribeHotBigKeys	Queries the hotkeys and large keys in the memory in real time.
DescribeTopHotKeys	Queries the top 20 hotkeys over a specific period of time.
DescribeTopBigKeys	Queries the top 20 large keys over a specific period of time.
DescribeHotKeys	Queries the hotkeys of an ApsaraDB for Redis instance.