Hybrid-storage instances of Tair store data in both memory and disks to support data persistence and high read and write performance at the same time.
Description
Hybrid-storage instances of Tair are developed by Alibaba Cloud and are compatible with the Redis protocol. These instances store all data in disks and hot data in memory to ensure high read and write performance. Hybrid-storage instances deliver a cost-effective solution that supports high-performance queries for frequently accessed data without the limit of memory capacity.
- In-memory storage: Keys and values of hot data are stored in memory. The information of all keys is also cached in memory. This way, you can check whether a specified key exists in a fast and convenient manner.
- Disk storage: All keys and values are stored in disks. Redis data structures such as hashes are also stored in disks in a specific format.
Scenarios
| Scenario | Description |
|---|---|
| Livestreaming | Livestreams generate large amounts of hot data. Most of the data comes from popular live channels. You can use hybrid-storage instances to store data from popular live channels in memory and data from less popular live channels in disks. This allows you to make full use of the limited memory capacity. |
| E-commerce | E-commerce applications need to access large amounts of commodity data. You can use hybrid-storage instances to store data and optimize memory usage. The data of popular commodities is stored in memory, and the data of less popular commodities is stored in disks. |
| Online education |
Online education applications need to access large amounts of data. Such data includes online courses, question libraries, and messages between teachers and students. Only popular courses and the latest question libraries are frequently accessed. You can use hybrid-storage instances to store the data of online courses in disks, and store the data of popular courses and question libraries in memory. Tair provides a cost-effective solution to ensure high read and write performance for frequently accessed data. |
The following examples show the benefits of hybrid-storage instances:
- Example 1: An open source Redis cluster is used to store 100 GB of data. The number
of queries per second (QPS) at peak hours is less than 20,000, and 80% of the data
is not frequently accessed.
In this case, you can use a Tair hybrid-storage instance that has 32 GB memory and 128 GB disk capacity. This saves about 70 GB of memory resources and reduces storage costs by more than 50%.
- Example 2: An on-premises Pika instance is used to reduce the storage costs of an
open source Redis deployment. The total size of the data is about 400 GB, and only
about 10% of the data is frequently accessed. In addition, high costs are incurred
for cluster O&M.
In this case, you can use a hybrid-storage instance that has 64 GB memory and 512 GB disk capacity. This reduces your O&M costs and ensures high availability.
Select instances based on scenarios
| Item | Hybrid-storage instance | Community Edition instance |
|---|---|---|
| Data size and budget |
|
You have a small amount of data, or your budget is sufficient for storage costs. |
| Separation of hot and cold data | Your business data can be divided into hot data and cold data.
Note Cold data is the data that is not frequently accessed.
|
Your business data is accessed at random. |
| QPS performance and latency requirements for hot and cold data |
|
|
| Access frequency of large keys |
|
Large keys are accessed at random and are sensitive to latency. |
Select instances based on specifications
When you create a hybrid-storage instance of Tair, you must specify the memory capacity and the disk capacity. The memory capacity determines the maximum amount of hot data that can be stored, and the disk capacity determines the total amount of data that can be stored. Tair creates an instance with the corresponding CPU resources based on the instance specifications that you choose. For example, if you select a master-replica instance that has 64 GB of memory and 256 GB of disk capacity, the instance can store up to 256 GB of data and can cache 64 GB of data in memory.
For more information, see Overview.
- You cannot specify the number of CPU cores.
- Hybrid-storage instances store the metadata of keys in memory and disks. The metadata includes the time-to-live (TTL) value, the least recently used (LRU) clock value, and the type of each key. When you specify the memory capacity and the disk capacity, you must reserve storage capacity for metadata. For more information, see the following description in this topic.
- Memory capacity
To support more open source Redis features, hybrid-storage instances store all keys in memory. These instances store the values of frequently accessed keys in memory and the values of all keys in disks. Therefore, make sure that the specified memory capacity is sufficient to store all keys and related metadata. The following table lists the recommended memory capacity based on the total number of keys.
Total number of keys Recommended memory capacity Less than 20 million 64 GB, 32 GB, and 16 GB 20 million to 50 million 64 GB and 32 GB 50 million to 100 million 128 GB, 64 GB, and 32 GB More than 100 million 128 GB and 64 GB Note The memory capacity that you specify when you create a Tair instance determines the CPU resources that are allocated to the instance. Instances that have a higher memory capacity provide higher performance. - Disk capacity
Hybrid-storage instances store all data in disks, including the metadata generated for each key. The metadata occupies extra storage. Therefore, we recommend that you select a disk capacity that is 20% to 50% more than the required storage.
Instance performance
The performance of a hybrid-storage instance depends on the instance type and memory hit ratio. The hit ratio indicates the probability that the requested data is found in memory. Higher specifications and higher hit ratios indicate higher performance. If all requested data is found in the memory of the hybrid-storage instance, the performance is the same as that of a Community Edition instance. A lower memory hit ratio indicates the lower performance of the hybrid-storage instance. If all requested data is found in disks, the hybrid-storage instance delivers the lowest performance.
In the following performance tests, each Tair instance stores 20 million keys and the size of each value is 1 KB. The GET command is used to read values. Different types of keys are accessed in the following scenarios.
| Testing scenario | QPS of a Community Edition instance | QPS of a hybrid-storage instance |
|---|---|---|
| Keys are accessed at random. | 123,000 | 15,000 |
| Keys are accessed based on the Gaussian distribution, and 20% of the keys are accessed at a probability of 80%. | 120,000 | 54,000 |
| Keys are accessed based on the Gaussian distribution, and 1% of the keys are accessed at a probability of 99%. | 135,000 | 114,000 |