ApsaraDB for Redis Enhanced Edition (Tair) supports two instance types: performance-enhanced instances and hybrid-storage instances. Different from instances of Community Edition, hybrid-storage instances store data in memory and disks. Hybrid-storage instances use the benefits of memory and disks to provide persistent data storage with high read and write performance.

Introduction

Figure 1. Architecture
Architecture

Hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair), which are independently developed by Alibaba Cloud, are compatible with the Redis protocol. These instances store all data in disks and hot data in memory to ensure high read and write performance. Hybrid-storage instances deliver a cost-effective solution to provide high-performance queries for frequently accessed data without the limit of memory capacity.

  • In-memory storage: Keys and values of hot data are stored in memory. The information about keys is also cached in memory. This allows you to check whether a specified key exists.
  • Disk storage: All keys and values are stored in disks. Redis data structures such as Hashes are also stored in disks by using a certain format.

Scenarios

Scenario Description
Live video streaming Live video streams generate a large amount of hot data. Most of the data is from popular live streaming rooms. You can use hybrid-storage instances to store data from popular live streams in memory and data from less popular live streams in disks. This allows you to make full use of the limited memory capacity.
E-commerce E-commerce applications generate a large amount of data. You can use hybrid-storage instances to store data and optimize memory usage. The data of popular items is stored in memory and the data of less popular items is stored in disks.
Online education

Online education applications generate a large amount of data. Such data includes online courses, question libraries, and messages between teachers and students. Only popular courses and the latest question libraries are frequently queried. You can use hybrid-storage instances to store the data of online courses in disks, and store the data of popular courses and question libraries in memory. ApsaraDB for Redis provides a cost-effective solution to ensure high read and write performance for frequently queried data.

The following examples show the benefits of hybrid-storage instances:

  • Example 1: A native Redis cluster is used to store 100 GB of data. The query rate is less than 20,000 queries per second (QPS) at peak hours and 80% of the data is infrequently accessed.

    In this case, you can use a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair) that has 32 GB memory and 128 GB disk capacity. This decreases memory usage by about 70 GB and reduces storage costs by more than 50%.

  • Example 2: An on-premises Pika instance is used to reduce the storage costs of a native Redis deployment. The total size of the data is about 400 GB and only about 10% of the data is frequently accessed. High costs are incurred for cluster operations and maintenance (O&M).

    In this case, you can use a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair) that has 64 GB memory and 512 GB disk capacity. This minimizes your investment for O&M and ensures high availability.

Select instances based on scenarios

Hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair) and cache-only instances of ApsaraDB for Redis Community Edition are applicable to different scenarios, as shown in the following table.
Scenario Hybrid-storage instances Instances of Community Edition
Data size and budget
  • You need to store a large amount of data at a lower cost.
  • You use Pika, SSDB, or Antibiotic Resistance Genes Database (ARDB) database services.
  • You use master-replica or cluster instances of ApsaraDB for Redis to store a large amount of data.
You need to store a small amount of data, or your budget is sufficient for storage costs.
Separation of hot and cold data Your business data is divided into hot data and cold data.
Note Cold data is the data that is infrequently accessed.
Your business data is accessed at random.
QPS performance and latency requirements for hot and cold data
  • High QPS performance is required for hot data.
  • The access to cold data is not affected by latency.
  • High QPS performance is required for all data.
  • The access to all data is affected by latency.
Access frequency of big keys
  • No big key exists in your business data.
  • Big keys are frequently accessed and need to be stored in memory.
  • Big keys are infrequently accessed and are not affected by latency.
Big keys are irregularly accessed and are highly sensitive to latency.

Select instances based on specifications

When you create a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair), you must specify the memory and disk capacity. The memory capacity specifies the maximum amount of hot data that can be stored and the disk capacity specifies the total amount of data that can be stored. ApsaraDB for Redis allocates CPU resources to the instance based on the specified memory and disk capacity. For example, if you specify a master-replica instance that has 64 GB of memory and 256 GB of disk capacity, the instance can store up to 256 GB of data and can cache 64 GB of data in memory.

For more information, see Overview.

Note
  • You cannot customize the number of CPU cores.
  • Hybrid-storage instances store the metadata of keys in memory and disks. The metadata includes the time-to-live (TTL) value, the least recently used (LRU) clock value, and the type of each key. When you specify the memory and disk capacity, you must reserve the storage capacity for metadata. For more information, see the following description in this topic.
  • Memory capacity

    To support more native Redis features, hybrid-storage instances store all keys in memory. These instances store the values of frequently accessed keys in memory and the values of infrequently accessed keys in disks. Therefore, make sure that the specified memory capacity is sufficient to store all keys and related metadata. The following table lists the recommended memory capacity based on the total number of keys.

    Total number of keys Recommended memory capacity
    Less than 20 million 64 GB, 32 GB, and 16 GB
    20 million to 50 million 64 GB and 32 GB
    50 million to 100 million 128 GB, 64 GB, and 32 GB
    More than 100 million 128 GB and 64 GB
    Note The memory capacity determines the CPU resources that are allocated to the instance when you create an instance. A higher memory capacity indicates higher performance of the instance.
  • Disk capacity

    Hybrid storage instances store all data in disks, including the metadata generated for each key. The metadata occupies extra storage. Therefore, we recommend that you select a disk capacity that is 20% to 50% more than the required storage.

Purchase an instance

For more information, see Step 1: Create an ApsaraDB for Redis instance.

Instance performance

The performance of a hybrid-storage instance depends on the instance type and memory hit ratio. This hit ratio indicates the probability that the requested data is found in memory. Higher specifications and a higher memory hit ratio indicate higher performance. If all requested data is found in the memory of the hybrid-storage instance, the performance is the same as that of a high-performance in-memory instance. A lower memory hit ratio indicates lower performance of the hybrid-storage instance. If all requested data is found in disks, the hybrid-storage instance delivers the lowest performance.

In the following performance tests, the ApsaraDB for Redis instance stores 20 million keys and the size of each value is 1 KB. The GET command is used to read values. Different types of keys are accessed in the following scenarios.

Table 1. Test results
Test scenario QPS of a Community Edition instance QPS of a hybrid-storage instance
Keys are accessed at random. 123,000 15,000
Based on the Gaussian distribution, 20% of the keys are accessed at a probability of 80%. 120,000 54,000
Based on the Gaussian distribution, 1% of the keys are accessed at a probability of 99%. 135,000 114,000

FAQ

  • Do Hybrid-storage instances have limits on commands?

    Hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair) support most native Redis commands. To ensure high performance, hybrid-storage instances impose limits on the use of some commands. For more information, see Limits on commands supported by Enhanced Edition.

  • What is the threshold of hot data in memory? What may happen after the threshold is exceeded?

    The threshold of hot data in memory is 90% of the memory capacity of the instance. For example, for a hybrid-storage instance with a memory capacity of 16 GB, if the amount of used memory exceeds 14.4 GB, ApsaraDB for Redis evicts values of cold data from memory. However, values of cold data are still stored in disks.

  • What is the granularity of data eviction if the amount of used memory exceeds the threshold?

    Keys are the smallest unit for eviction. For example, if the memory usage of a hash exceeds the threshold, the hash table is evicted.

  • How is a key-value pair stored in memory?

    Compared with instances of Community Edition, hybrid-storage instances store additional metadata. The metadata is used to record whether the values of keys are cached in memory. It can also be used to check whether the data is hot or cold when it is evicted.

  • How is data written to disks?
    To avoid direct access to disks, data is first modified in memory. This is similar to the process for page cache. Then, background threads continuously synchronize data modifications to the disks.
    Note Hot data can be directly modified in memory. Cold data must be loaded to memory before modification.
  • How does a hybrid-storage instance work if the data to be accessed is not cached in memory?
    ApsaraDB for Redis checks whether the data to be accessed is cached in memory before it runs a command. If the data is not cached in memory, ApsaraDB for Redis caches the data from disks to memory before it runs the command.
    Note It takes longer to load data with more complex data structures from disks. We recommend that you evaluate access models and data models to avoid frequent requests to cold data.
  • Why does a write timeout occur?
    The write timeout may occur due to the following issues:
    • When the cold data is requested, it takes a long time to cache data from disks to the memory.
    • A large number of concurrent write requests are processed. After the data is modified in memory, background threads synchronize the modification to disks. However, the write speed of disks is lower than that of the memory. When the data is written to memory at a much faster rate than to disks, a write speed limit must be set on the memory to prevent write timeouts.

Other FAQ

  • An OOM error occurred when I wrote data to a hybrid-storage instance. All memory is consumed, but the disks still have some remaining capacity. How can I fix this issue?

    The memory capacity is insufficient to store all keys and related metadata. Therefore, the memory capacity must be increased. For more information, see Change specifications.

  • Can I change configurations from hybrid-storage instances to Community Edition instances or performance-enhanced instances.

    No, you cannot change the configurations from hybrid-storage instances to Community Edition instances or performance-enhanced instances.

  • How do I migrate data from hybrid-storage instances to Community Edition instances or performance-enhanced instances?
    You can use the redis-shake tool to migrate full data. For more information, see Use redis-shake to migrate data.
    Note
    • Hybrid storage instances use different encoding formats and replication protocols from Community Edition instances. You can use only the rump or sync mode to migrate full data. The migration of incremental data is not supported.
    • You cannot use Data Transmission Service (DTS) to migrate data.
  • How do I export data from a hybrid storage instance to on-premises storage?

    You can use the dump mode of the redis-shake tool to export Redis Database Backup (RDB) files. If you use the Community Edition replication protocol to connect to a hybrid-storage instance, the hybrid-storage instance converts data to the RDB format and sends the data to the redis-shake tool. For more information, see Use the redis-shake tool to back up data.

  • Can I mount a Community Edition instance or a performance-enhanced instance of Enhanced Edition (Tair) to a hybrid-storage instance as a secondary instance?

    Yes, you can perform this operation. This operation is applicable only in specific scenarios because hybrid-storage instances support only full data synchronization.