To ensure instance stability and data consistency, you must take note of the limits described in this topic when you use the Global Distributed Cache feature.
Usage notes
Each child instance must be a DRAM-based instance.
Make sure that the specifications of each child instance are the same. If you want to change the specifications of a child instance, we recommend that you change all the other child instances to the same specifications. If the specifications of the child instances in a distributed instance are not all the same, performance or capacity issues may occur.
A distributed instance can contain up to three child instances, and the child instances cannot be located in the same zone. The first child instance can be converted from an existing instance. The second and third child instances must be newly purchased instances.
All child instances of a distributed instance must either reside in the Chinese mainland or be situated in regions outside the Chinese mainland. If you want to synchronize data from the Chinese mainland to regions outside the Chinese mainland, use Data Transmission Service (DTS). For more information, see Apply for permissions to synchronize data across borders.
A child instance of a distributed instance has the following limits:
You cannot change the zone of the instance.
You cannot change the architecture of the instance, for example, from the cluster architecture to the standard architecture.
When you change the configurations of a cluster instance, you can adjust either the shard specifications or the number of shards. For more information, see Why am I unable to change the configurations of a classic (local disk-based) cluster instance?
Limits on commands that can be synchronized
The FLUSHDB or FLUSHALL command cannot be synchronized.
To clear data, you must run the FLUSHALL command on all child instances. Otherwise, data inconsistency occurs.
To resynchronize data, for example, from Instance A to Instance B, you must remove Instance B from the distributed instance, run the FLUSHALL command on Instance B, and then reconnect Instance B to the distributed instance.
The Pub and Sub commands cannot be synchronized. To replicate notification messages across domains, we recommend that you use the Stream data structure.
The RESTORE command that is synchronized to the destination instance can be run only when the destination instance does not contain a key with the same name as the specified one.
DEL commands generated by data eviction and expiration are not synchronized.
Limits on synchronization granularity
Data is synchronized at the instance level. In other words, the data of each child instance must be synchronized as a whole. Global Distributed Cache cannot synchronize only some of the data from a child instance to another child instance.
If you want to synchronize only some of the data in a child instance, you can modify your business logic.
Limits on data consistency
In scenarios where one child instance is writable, such as disaster recovery scenarios, the final data must be consistent on all child instances. Data inconsistency may occur in the following situations:
Operations performed on keys with the specified time to live (TTL) may cause data inconsistency due to reasons such as synchronization latency. For example, the EXPIRE command is successfully run on Child Instance A to renew a key, but data has expired when it is synchronized to Child Instance B. If you require high data consistency, we recommend that you do not specify a TTL for a Global Distributed Cache instance.
The keys for data eviction in each child instance are independent and randomly selected. Make sure that the memory usage does not exceed the instance specifications to avoid triggering data eviction.
In scenarios where multiple child instances are writable, such as active geo-redundancy scenarios, make sure that your business system does not modify the same key in multiple child instances at the same time or in a short interval. Otherwise, Global Distributed Cache cannot ensure that the final values of the key are the same in each child instance, and data may pile up in child instances in streaming scenarios where conflicts can arise due to strict monotonicity. The following table describes the types of data inconsistency that may occur.
NoteGlobal Distributed Cache does not support conflict-free replicated data types (CRDTs). You must ensure data consistency in your business system.
Inconsistency
Illustration
Description
Failed key renewal
At point in time 1, Child Instances A and B hold a key with the specified TTL. The key expires at point in time 3.
At point in time 2, Child Instance A runs the PEXPIREAT command to renew the key to point in time 5.
At point in time 3, Child Instance B does not receive the PEXPIREAT synchronization command from Child Instance A due to network latency. At the same point in time, the key in Child Instance B is deleted due to expiration.
At point in time 4, Child Instance B receives the extension command for the key, but the key is deleted due to expiration. Therefore, the extension command fails to be run. In this case, Child Instance A still holds the key, and the key in Child Instance B is lost.
Inconsistent command execution results due to the TTL
The SMOVE command is used in the following example.
At point in time 1, Child Instances A and B hold Key1 and Key2. Key1 expires at point in time 3, and Key2 does not have a TTL configured.
At point in time 2, Child Instance A runs the
SMOVE Key1 Key2 "foo"command to move the "foo" member to Key2.At point in time 3, Child Instance B does not receive the synchronization command from Child Instance A due to reasons such as network latency. At the same point in time, Key1 in Child Instances A and B is deleted due to expiration.
At point in time 4, Child Instance B receives the SMOVE command, but the SMOVE command fails to be run because Key1 is deleted due to expiration. In this case, Key2 in Child Instance A contains the "foo" member, but Key2 in Child Instance B does not contain the "foo" member.
Random data eviction
At point in time 1, Child Instances A and B hold Key1 and Key2.
At point in time 2, the memory of Child Instance A is exhausted, data eviction is triggered, and Key2 is randomly deleted.
At point in time 3, the memory of Child Instance B is exhausted, data eviction is triggered, and Key1 is randomly deleted.
The default data eviction policy is volatile-lru, and deletion operations during data eviction are not synchronized. Therefore, the data of Child Instances A and B is inconsistent.
Data writes to multiple child instances - value swapping
If a key is written to multiple child instances at the same time, data may be inconsistent.
At point in time 1,
Key:valAis written to Child Instance A.At point in time 2,
Key:valBis written to Child Instance B.At point in time 3, Child Instance A synchronizes
Key:valAto Child Instance B. At the same point in time, Child Instance B synchronizesKey:valBto Child Instance A.After the synchronization is complete, the values of the key are swapped between the two child instances.
Data writes to multiple child instances - inconsistent data types
At point in time 1, a
Keyof the Hash data type is written to Child Instance A.At point in time 2, a
Keyof the String data type is written to Child Instance B.When a synchronization command is run at point in time 3, a failure occurs due to data type conflicts.
Data writes to multiple child instances - unsatisfied write conditions
At point in time 1,
SETNX Key valAis written to Child Instance A.At point in time 2,
SETNX Key valBis written to Child Instance B.When a synchronization command is run at point in time 3, a failure occurs due to unsatisfied write conditions.
NoteCommands, such as HSETNX and SET[NX | XX], also cause similar issues.
Data writes to multiple child instances - disordered data or data loss
At point in time 1,
RPUSH Key valAis written to Child Instance A.At point in time 2,
RPUSH Key valBis written to Child Instance B.When a synchronization command is run at point in time 3, data is out of order or lost.
NoteCommands, such as LPUSH, APPEND, DEL, HDEL, INCR, and XADD, also cause similar issues.