ApsaraDB for Redis Enhanced Edition (Tair) supports performance-enhanced instances and hybrid-storage instances. Different from instances of Community Edition, hybrid-storage instances store data in both memory and disks. Hybrid-storage instances combine the benefits of memory and disks to support data persistence and high read and write performance at the same time.

Note Hybrid-storage instances are discontinued. For more information, see ApsaraDB for Redis hybrid-storage instances are discontinued. We recommend that you use other types of ApsaraDB for Redis Enhanced Edition (Tair) instances, such as Performance-enhanced instances, Persistent memory-optimized instances, and Storage-optimized instances.

Overview

Figure 1. Architecture
Architecture

Hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair) are developed by Alibaba Cloud and are compatible with the Redis protocol. These instances store all data in disks and hot data in memory to ensure high read and write performance. Hybrid-storage instances deliver a cost-effective solution that supports high-performance queries for frequently accessed data without the limit of memory capacity.

  • In-memory storage: Keys and values of hot data are stored in memory. The information about all keys is also cached in memory. This way, you can check whether a specified key exists in a fast and convenient manner.
  • Disk storage: All keys and values are stored in disks. Redis data structures such as hashes are also stored in disks by using a specific format.

Scenarios

Scenario Description
Live streaming Live streams generate a large amount of hot data. Most of the data comes from popular live channels. You can use hybrid-storage instances to store data from popular live channels in memory and data from less popular live channels in disks. This allows you to make full use of the limited memory capacity.
E-commerce E-commerce applications need to access a large amount of commodity data. You can use hybrid-storage instances to store data and optimize memory usage. The data of popular items is stored in memory and the data of less popular items is stored in disks.
Online education

Online education applications need to access a large amount of data. Such data includes online courses, question libraries, and messages between teachers and students. Only popular courses and the latest question libraries are frequently accessed. You can use hybrid-storage instances to store the data of online courses in disks, and store the data of popular courses and question libraries in memory. ApsaraDB for Redis provides a cost-effective solution to ensure high read and write performance for frequently accessed data.

The following examples show the benefits of hybrid-storage instances:

  • Example 1: A native Redis cluster is used to store 100 GB of data. The queries per second (QPS) at peak hours is less than 20,000 and 80% of the data is not frequently accessed.

    In this case, you can use a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair) that has 32 GB memory and 128 GB disk capacity. This decreases memory usage by about 70 GB and reduces storage costs by more than 50%.

  • Example 2: An on-premises Pika instance is used to reduce the storage costs of a native Redis deployment. The total size of the data is about 400 GB and only about 10% of the data is frequently accessed. High costs are incurred for cluster O&M.

    In this case, you can use a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair) that has 64 GB memory and 512 GB disk capacity. This reduces your O&M costs and ensures high availability.

Select instances based on scenarios

Hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair) and memory-only instances of ApsaraDB for Redis Community Edition are applicable to different scenarios, as shown in the following table.
Scenario Hybrid-storage instance Community Edition instance
Data size and budget
  • You have a large amount of data to store and want to reduce the storage costs.
  • You are using Pika, SSDB, or Antibiotic Resistance Genes Database (ARDB) services.
  • You are using master-replica or cluster instances of ApsaraDB for Redis to store a large amount of data.
You have a small amount of data, or your budget is sufficient for storage costs.
Separation of hot and cold data Your business data can be divided into hot data and cold data.
Note Cold data is the data that is not frequently accessed.
Your business data is accessed at random.
QPS performance and latency requirements for hot and cold data
  • High QPS performance is required for hot data.
  • The access to cold data is not sensitive to latency.
  • High QPS performance is required for all data.
  • The access to all data is sensitive to latency.
Access frequency of big keys
  • No big key exists in your business data.
  • Big keys are frequently accessed and need to be stored in memory.
  • Big keys are not frequently accessed and are not sensitive to latency.
Big keys are accessed at random and are sensitive to latency.

Select instances based on specifications

When you create a hybrid-storage instance of ApsaraDB for Redis Enhanced Edition (Tair), you must specify the memory capacity and the disk capacity. The memory capacity determines the maximum amount of hot data that can be stored and the disk capacity determines the total amount of data that can be stored. ApsaraDB for Redis allocates CPU resources to the instance based on the specified memory capacity and disk capacity. For example, if you select a master-replica instance that has 64 GB of memory and 256 GB of disk capacity, the instance can store up to 256 GB of data and can cache 64 GB of data in memory.

For more information, see Overview.

Note
  • You cannot customize the number of CPU cores.
  • Hybrid-storage instances store the metadata of keys in memory and disks. The metadata includes the time-to-live (TTL) value, the least recently used (LRU) clock value, and the type of each key. When you specify the memory capacity and the disk capacity, you must reserve storage capacity for metadata. For more information, see the following description in this topic.
  • Memory capacity

    To support more native Redis features, hybrid-storage instances store all keys in memory. These instances store the values of frequently accessed keys in memory and the values of all keys in disks. Therefore, make sure that the specified memory capacity is sufficient to store all keys and related metadata. The following table lists the recommended memory capacity based on the total number of keys.

    Total number of keys Recommended memory capacity
    Less than 20 million 64 GB, 32 GB, and 16 GB
    20 million to 50 million 64 GB and 32 GB
    50 million to 100 million 128 GB, 64 GB, and 32 GB
    More than 100 million 128 GB and 64 GB
    Note The memory capacity that you specify when you create an ApsaraDB for Redis instance determines the CPU resources that are allocated to the instance. Instances that have higher memory capacities provide higher performance.
  • Disk capacity

    Hybrid-storage instances store all data in disks, including the metadata generated for each key. The metadata occupies extra storage. Therefore, we recommend that you select a disk capacity that is 20% to 50% more than the required storage.

Purchase and activate an ApsaraDB for Redis instance

For more information, see Step 1: Create an ApsaraDB for Redis instance.

Instance performance

The performance of a hybrid-storage instance depends on the instance type and memory hit ratio. The hit ratio indicates the probability that the requested data is found in memory. Higher specifications and higher hit ratios indicate higher performance. If all requested data is found in the memory of the hybrid-storage instance, the performance is the same as that of a Community Edition instance. A lower memory hit ratio indicates lower performance of the hybrid-storage instance. If all requested data is found in disks, the hybrid-storage instance delivers the lowest performance.

In the following performance tests, each ApsaraDB for Redis instance stores 20 million keys and the size of each value is 1 KB. The GET command is used to read values. Different types of keys are accessed in the following scenarios.

Table 1. Test results
Testing scenario QPS of a Community Edition instance QPS of a hybrid-storage instance
Keys are accessed at random. 123,000 15,000
Keys are accessed based on the Gaussian distribution, and 20% of the keys are accessed at a probability of 80%. 120,000 54,000
Keys are accessed based on the Gaussian distribution, and 1% of the keys are accessed at a probability of 99%. 135,000 114,000

FAQ about features

  • Q: Do hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair) have limits on commands?

    A: Hybrid-storage instances of ApsaraDB for Redis Enhanced Edition (Tair) support most native Redis commands. To ensure high performance, hybrid-storage instances impose limits on the use of specific commands. For more information, see Limits on commands supported by ApsaraDB for Redis Enhanced Edition (Tair).

  • Q: What is the threshold of used memory? What happens if the threshold is exceeded?

    A: The threshold is 90% of the memory capacity of the instance. For example, if the amount of used memory exceeds 14.4 GB for a hybrid-storage instance with a memory capacity of 16 GB, ApsaraDB for Redis evicts values of less frequently accessed keys from memory. The values of such keys are still stored in disks.

  • Q: What is the granularity of data eviction if the amount of used memory exceeds the threshold?

    A: Keys are the smallest unit for eviction. For example, if the memory usage of a hash exceeds the threshold, the entire hash table is evicted.

  • Q: How is a key-value pair stored in memory?

    A: Compared with instances of Community Edition, hybrid-storage instances store additional metadata. The metadata specifies whether the values of keys are cached in memory. The metadata can also be used to check whether data is hot or cold when the data is evicted.

  • Q: How is data written to disks?
    A: To prevent direct access to disks, ApsaraDB for Redis modifies data in memory and then uses background threads to continuously synchronize the modifications to disks. This is similar to the process for a page cache.
    Note Hot data is directly modified in memory. Cold data must be loaded to memory before modification.
  • Q: How does a hybrid-storage instance work if the requested data is not cached in memory?
    A: ApsaraDB for Redis checks whether the requested data is cached in memory before it runs a command. If the data is not cached in memory, ApsaraDB for Redis loads the data from disks to memory before it runs the command.
    Note It takes longer to load data with more complex data structures from disks. We recommend that you evaluate access models and data models to prevent frequent requests to cold data.
  • Q: Why does a write timeout occur?
    A: A write timeout may occur due to the following issues:
    • When the cold data is requested, it takes an extended period of time to load data from disks to the memory.
    • A large number of concurrent write requests are processed. After the data is modified in memory, background threads synchronize the modifications to disks. However, the write speed of disks is lower than that of the memory. When the data is written to memory at a much faster rate than to disks, a write speed limit must be set on the memory to prevent write timeouts.

Other FAQ

  • Q: An out of memory (OOM) error occurred when I wrote data to a hybrid-storage instance. All memory is consumed, but the disks still have free space. How can I fix this issue?

    A: The memory capacity is insufficient to store all keys and related metadata. Therefore, the memory capacity must be increased. For more information, see Change the configurations of an instance.

  • Q: Can I change configurations from hybrid-storage instances to Community Edition instances or performance-enhanced instances?

    A: No, you cannot change the configurations from hybrid-storage instances to Community Edition instances or performance-enhanced instances.

  • Q: How do I migrate data from hybrid-storage instances to Community Edition instances or performance-enhanced instances?
    A: You can use the redis-shake tool to migrate full data. For more information, see Migrate data between ApsaraDB for Redis instances by using Redis-shake.
    Note
    • Hybrid storage instances use different encoding formats and replication protocols from Community Edition instances. You can use only the rump or sync mode to migrate full data. The migration of incremental data is not supported.
    • You cannot use Data Transmission Service (DTS) to migrate data.
  • Q: How do I export data from a hybrid storage instance to local storage?

    A: You can use the redis-shake tool to export Redis Database Backup (RDB) files in dump mode. If you use the Community Edition replication protocol to connect to a hybrid-storage instance, the hybrid-storage instance converts data to the RDB format and sends the data to the redis-shake tool. For more information, see Use the redis-shake tool to back up data.

  • Q: Can I attach a Community Edition instance or a performance-enhanced instance of ApsaraDB for Redis Enhanced Edition (Tair) to a hybrid-storage instance as a secondary instance?

    A: Yes, you can perform this operation in specific scenarios. Hybrid-storage instances support only full data synchronization.