ApsaraDB for Redis Enhanced Edition (Tair) supports two instance types: performance-enhanced instances and hybrid-storage instances. Hybrid-storage instances store data in memory and disks whereas in-memory instances store all data in memory. Hybrid-storage instances combine the benefits of memory and disks to provide high read/write performance and persistent data storage.
Hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair), which are independently developed by Alibaba Cloud, are compatible with the Redis protocol. These instances store all data in disks and hot data in memory to ensure high read/write performance. Hybrid-storage instances deliver a cost-effective solution to provide high-performance queries for frequently accessed data without the limit of memory capacity.
- In-memory storage: Keys and values of hot data are stored in memory. The information about keys is also cached in memory. This allows you to check whether a specified key exists.
- Disk storage: All keys and values are stored in disks. Redis data structures such as Hash are also stored in disks in a certain format.
- Live video streaming
Live video streams generate a large amount of hot data. This type of data is from popular live streaming rooms. You can use hybrid-storage instances to store data from popular live streams in memory and data from less popular live streams in disks. This allows you to make full use of the limited memory capacity.
E-commerce applications generate a large amount of data. You can use hybrid-storage instances to store data and optimize memory usage. The data of popular items is stored in memory and data of less popular items is stored in disks.
- Online education
Online education applications generate a large amount of data. Such data includes online courses, question libraries, and messages between teachers and students. Only popular courses and the latest question libraries are frequently queried. You can use hybrid-storage instances to store the data of online courses in disks, and store the data of popular courses and question libraries in memory. ApsaraDB for Redis provides a cost-effective solution to ensure high read/write performance for frequently queried data.
The following examples show the benefits of hybrid-storage instances:
- Example 1: A native Redis cluster is used to store 100 GB of data. The query rate is less than 20,000 queries per second (QPS) at peak hours and 80% of the data is infrequently accessed. In this case, you can use a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair) that has 32 GB memory and 128 GB disk capacity. This decreases memory usage by about 70 GB and reduces storage costs by more than 50%.
- Example 2: An on-premises Pika instance is used to reduce the storage costs of a native Redis deployment. The total size of the data is about 400 GB and only about 10% of the data is frequently accessed. High costs are incurred for cluster operations and maintenance (O&M). In this case, you can use a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair) that has 64 GB memory and 512 GB disk capacity. This minimizes your O&M investment and ensures high availability.
Select instances based on scenarios
|Scenario||Hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair)||ApsaraDB for Redis Community Edition instance|
|Data size and budget||
||You need to store a small amount of data, or your budget is sufficient for storage costs.|
|Separation of hot and cold data||Your business data is divided into hot data and cold data.
Note Cold data is the data that is infrequently accessed.
|Your business data is accessed at random.|
|QPS performance and latency requirements for hot and cold data||
|Access frequency of big keys||
||Big keys are irregularly accessed and are highly sensitive to latency.|
Select instances based on specifications
When you create a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair), you must specify the memory and disk capacity. The memory capacity determines the maximum amount of hot data that can be stored and the disk capacity determines the total amount of data that can be stored. ApsaraDB for Redis allocates CPU resources to the instance based on the specified memory and disk capacity. For example, if you specify a master-replica instance that has 64 GB memory and 256 GB disk capacity, the instance can store up to 256 GB of data and can cache 64 GB of data in memory.
For more information, see Overview.
- You cannot customize the number of CPU cores.
- Hybrid-storage instances store the metadata of keys in memory and disks. The metadata includes the time-to-live (TTL) value, the least recently used (LRU) clock value, and the type of each key. When you specify the memory and disk capacity, you must reserve the storage capacity for metadata. For more information, see the following description in this topic.
- Memory capacity
To support more native Redis features, hybrid-storage instances store all keys in memory. These instances store the values of frequently accessed keys in memory and the values of infrequently accessed keys in disks. Therefore, make sure that the specified memory capacity is sufficient to store all keys and related metadata. The following table lists the recommended memory capacity based on the total number of keys.
Total number of keys Recommended memory capacity Less than 20 million 64 GB, 32 GB, and 16 GB 20 million to 50 million 64 GB and 32 GB 50 million to 100 million 128 GB, 64 GB, and 32 GB More than 100 million 128 GB and 64 GBNote The memory capacity determines the CPU resources that are allocated to the instance when you create an instance. A higher memory capacity indicates higher performance of the instance.
- Disk capacity
Hybrid storage instances store all data in disks, including the metadata generated for each key. The metadata occupies extra storage. Therefore, we recommend that you select a disk capacity that is 20% to 50% more than the required storage.
Purchase an instance
For more information, see Step 1: Create an ApsaraDB for Redis instance.
The performance of a hybrid-storage instance depends on the instance type and memory hit ratio. This hit ratio indicates the probability that the requested data is found in memory. Higher specifications and a higher memory hit ratio indicate higher performance. If all requested data is found in the memory of the hybrid-storage instance, the performance is the same as that of a high-performance in-memory instance. A lower memory hit ratio indicates lower performance of the hybrid-storage instance. If all requested data is found in disks, the hybrid-storage instance delivers the lowest performance.
In the following performance tests, the ApsaraDB for Redis instance stores 20 million keys and the size of each value is 1 KB. The GET command is used to read values. Different types of keys are accessed in the following scenarios.
|Test scenario||QPS of a Community Edition instance||QPS of a hybrid-storage instance|
|Keys are accessed at random.||123,000||15,000|
|Based on the Gaussian distribution, 20% of the keys are accessed at a probability of 80%.||120,000||54,000|
|Based on the Gaussian distribution, 1% of the keys are accessed at a probability of 99%.||135,000||114,000|
Hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair) support most native Redis commands. To ensure high performance, hybrid-storage instances impose limits on the use of some commands. For more information, see Commands supported by hybrid-storage instances.
- What is the threshold of hot data in memory? What may happen after the threshold is
The threshold of hot data in memory is 90% of the memory capacity of the instance. For example, for a hybrid-storage instance with a memory capacity of 16 GB, if the amount of used memory exceeds 14.4 GB, ApsaraDB for Redis evicts values of cold data from memory. However, values of cold data are still stored in disks.
- What is the granularity of data eviction if the amount of used memory exceeds the
Keys are the smallest unit for eviction. For example, if the memory usage of a hash exceeds the threshold, the hash table is evicted.
- How is a key-value pair stored in memory?
Compared with instances of Community Edition, hybrid-storage instances store additional metadata. The metadata is used to record whether the values of keys are cached in memory, and check whether the data is hot or cold when it is evicted.
- How is data written to disks?
To avoid direct access to disks, data is first modified in memory, which is similar to page cache. Then, background threads continuously synchronize data modifications to the disks.Note Hot data can be directly modified in memory. Cold data must be loaded to memory before modification.
- How does a hybrid-storage instance work if the data to be accessed is not cached in
ApsaraDB for Redis checks whether the data to be accessed is cached in memory before it runs a command. If the data is not cached in memory, ApsaraDB for Redis caches the data from disks to memory before it runs the command.Note It takes longer to load data with more complex data structures from disks. We recommend that you evaluate access models and data models to avoid frequent requests to cold data.
- Why does a write timeout occur?
The write timeout may occur due to the following causes:
- When the cold data is requested, it takes a long time to cache data from disks to the memory.
- A large number of concurrent write requests are processed. After the data is modified in memory, background threads synchronize the modification to disks. However, the write speed of disks is less than that of the memory. When the data is written to memory at a much faster rate than to disks, a write speed limit must be set on the memory to prevent write timeouts.
OOM erroroccurred when I wrote data to a hybrid-storage instance. All memory is consumed, but the disks still have some remaining capacity. How can I resolve this issue?
The memory capacity is insufficient to store all keys and related metadata. Therefore, the memory capacity must be increased. For more information, see Change specifications.
- Can I change configurations from hybrid-storage instances to Community Edition instances
or performance-enhanced instances.
No, you cannot change the configurations from hybrid-storage instances to Community Edition instances or performance-enhanced instances.
- How do I migrate data from hybrid-storage instances to Community Edition instances
or performance-enhanced instances?
You can use the redis-shake tool to migrate full data. For more information, see Use redis-shake to migrate data.Note
- Hybrid storage instances use different encoding formats and replication protocols from Community Edition instances. You can use only the rump or sync mode to migrate full data. The migration of incremental data is not supported.
- You cannot use Data Transmission Service (DTS) to migrate data.
- How do I export data from a hybrid storage instance to my local device?
You can use the dump mode of the redis-shake tool to export Redis Database Backup (RDB) files. If you use the Community Edition replication protocol to connect to a hybrid-storage instance, the hybrid-storage instance converts data to the RDB format and sends the data to the redis-shake tool. For more information, see Use the redis-shake tool to back up data.
- Can I mount a Community Edition instance or a performance-enhanced instance of Enhanced
Edition (Tair) to a hybrid-storage instance as a secondary instance?
Yes, you can perform this operation. This operation is applicable only in specific scenarios because hybrid-storage instances support only full data synchronization.