Data Transmission Service:Use OpenAPI to configure one-way or two-way data synchronization for Tair instances

Prerequisites

Usage notes

• For two-way synchronization, the forward synchronization job performs initial full data synchronization and incremental data synchronization. The reverse synchronization job performs only incremental data synchronization.

  Warning

  To ensure data consistency, do not modify or write data to the same key in the source and destination databases when the two-way synchronization task is running.

• DTS uses the resources of the source and destination databases during initial full data synchronization. This may increase the loads on the database servers. If you synchronize a large amount of data or if the server specifications do not meet your requirements, database services may become unavailable. Before you synchronize data, evaluate the impact of data synchronization on the performance of the source and destination instances. We recommend that you synchronize data during off-peak hours.

• Do not execute FLUSHDB or FLUSHALL commands on the source instance. Otherwise, this will cause data inconsistency between the source and target.

• If the destination database runs out of memory and triggers data eviction, data may become inconsistent between the source and destination. This occurs because the default data eviction policy (maxmemory-policy) for Tair (Redis OSS-Compatible) is volatile-lru. However, this does not affect normal task operation.

  To prevent this issue, we recommend setting the data eviction policy for the destination database to noeviction. With this policy, if the destination runs out of memory, write operations fail, the task stops, and no data is lost to eviction.

  Note

  For more information about data eviction policies, see Data eviction policies in Redis.

• If some keys in the source database are set to expire, they might not be deleted immediately upon expiration. As a result, the key count in the destination database, which you can check using the info command, might be lower than in the source database.

  Note

  The number of keys that do not have an expiration policy or have not expired is the same in the source and destination databases.

• If direct connection is disabled for the destination Tair (Redis OSS-Compatible) Enhanced Edition instance, DTS uses the proxy forwarding mode to write data to the destination instance.

  Note

  For more information about how to enable direct connection, see Enable direct connection.

• If the source Redis instance is scaled (for example, shards are added or removed) or its specifications are changed (for example, memory is increased) during synchronization, you must reconfigure the task. To ensure data consistency, we recommend clearing all synchronized data from the destination Redis instance before you reconfigure the task.

• If the connection address of the source Redis instance changes during synchronization (for example, due to a change of availability zone or a network type switch from classic network to a Virtual Private Cloud (VPC)), you must reconfigure the task.

• The following limitations apply when you synchronize data from a standalone Redis instance to a cluster-mode Redis instance. A Redis cluster only allows a command to operate on keys that are in the same slot. If you perform a multi-key operation on the source database where the keys span multiple slots, the following error occurs:

  CROSSSLOT Keys in request don't hash to the same slot

  We recommend that you perform only single-key operations during data synchronization to prevent task interruptions.

• If the destination instance is a cluster instance and one of its shards reaches its memory limit, or if the destination instance runs out of storage space, the DTS job fails with an Out of Memory (OOM) error.

• If the transparent data encryption (TDE) feature is enabled for the source or destination instance, you cannot use DTS to synchronize data from the source database to the destination database.

• If the source instance is a self-managed Redis, we recommend that you set the replication timeout between the slave and master (repl-timeout) to 600 seconds by using the config set repl-timeout 600 command. If the source database contains a large volume of data, you can increase the value of the repl-timeout parameter.

• A two-way synchronization task includes forward and reverse synchronization tasks. When you configure or reset the task, if the destination object of one task matches the synchronization object of the other task:

  • Allow only one task to synchronize full and incremental data. The other task supports only incremental synchronization.

  • Data from the source of the current task synchronizes only to the destination of the current task. It does not serve as source data for the other task.

Billing

Limits

One-way cascade synchronization is not supported. For more information, see Synchronization topologies.

Commands that can be synchronized

• APPEND

• BITOP, BLPOP, BRPOP, and BRPOPLPUSH

• DECR, DECRBY, and DEL

• EVAL, EVALSHA, EXEC, EXPIRE, and EXPIREAT

• GEOADD and GETSET

• HDEL, HINCRBY, HINCRBYFLOAT, HMSET, HSET, and HSETNX

• INCR, INCRBY, and INCRBYFLOAT

• LINSERT, LPOP, LPUSH, LPUSHX, LREM, LSET, and LTRIM

• MOVE, MSET, MSETNX, and MULTI

• PERSIST, PEXPIRE, PEXPIREAT, PFADD, PFMERGE, and PSETEX

• RENAME, RENAMENX, RPOP, RPOPLPUSH, RPUSH, and RPUSHX

• SADD, SDIFFSTORE, SELECT, SET, SETBIT, SETEX, SETNX, SETRANGE, SINTERSTORE, SMOVE, SPOP, SREM, and SUNIONSTORE

• UNLINK, ZADD, ZINCRBY, ZINTERSTORE, ZREM, ZREMRANGEBYLEX, ZUNIONSTORE, ZREMRANGEBYRANK, and ZREMRANGEBYSCORE

• SWAPDB (This command cannot be synchronized if the source or destination instance is deployed in the cluster architecture.)

Before you begin

Create an AccessKey pair.

Create a data synchronization instance

1. Log on to OpenAPI Explorer.

2. Configure the request parameters based on your business requirements. The following table describes the parameters.

   Parameter	Description	Value
   SourceRegion	The ID of the region in which the source instance resides.	Specify a value for these parameters based on your business requirements.
   DestRegion	The ID of the region in which the destination instance resides.
   Topology	The synchronization topology. Valid values: oneway (default): one-way synchronization. bidirectional: two-way synchronization.
   SynchronizationJobClass	The instance class of the data synchronization instance.
   PayType	The billing method of the data synchronization instance. Valid values: PrePaid: subscription. PostPaid (default): pay-as-you-go.
   SourceEndpoint.InstanceType	The type of the source instance.	Set the value to Redis.
   DestinationEndpoint.InstanceType	The type of the destination instance.	Set the value to Redis.

   Note

   For more information, see CreateSynchronizationJob.

3. In the lower part of the page, click Initiate Call.

4. On the Response tab, view the API response.

   Note

   Record the synchronization instance ID (SynchronizationJobId) from the response.

   Important

   If the request fails, you need to modify the request parameters based on the message returned in the response and send the request again until the request is successful.

5. Optional: Get the SDK sample code.

   1. Click the SDK Sample Code tab.

   2. Specify the SDK version and programming language.

   3. Click the copy icon

Configure a one-way synchronization task

1. Log on to OpenAPI Explorer.

2. Configure the request parameters based on your business requirements.

   Note

   You must set the SynchronizationDirection parameter to Forward. For more information, see ConfigureSynchronizationJob.

3. In the lower part of the page, click Initiate Call.

4. On the Response tab, view the API response.

   Important

   If the call fails, check the Message in the response, fix the issue, and try again.

5. Optional. Obtain the sample code of the SDK.

   1. Click the SDK Sample Code tab.

   2. Specify the SDK version and programming language.

   3. Click the copy icon

Configure two-way synchronization tasks

In this example, the database accounts used for data synchronization have the read and write permissions.

1. Configure a data synchronization task in the forward direction. For more information, see the Configure a one-way synchronization task section of this topic.

2. Wait until the forward synchronization job completes initialization and its status changes to Synchronizing.

   Note

   You can view the status of a data synchronization task in the DTS console or by calling the Query the status of a data synchronization job operation.

3. Configure the data synchronization task in the reverse direction.

   1. Optional. Log on to OpenAPI Explorer.

   2. Configure the request parameters based on your business requirements.

      Note

      You must set the SynchronizationDirection parameter to Reverse. For more information, see ConfigureSynchronizationJob.

   3. In the lower part of the page, click Initiate Call.

   4. On the Response tab, view the API response.

      Important

      If the call fails, check the Message in the response, fix the issue, and try again.

   5. Optional. Obtain the sample code of the SDK.

      1. Click the SDK Sample Code tab.

      2. Specify the SDK version and programming language.

      3. Click the copy icon

References

• Synchronization task status and performance

  You can view the performance of data synchronization by using the performance metrics. DTS provides the connection and performance metrics to help you manage data synchronization tasks.

• Add synchronization objects or Remove an object from a data synchronization task

  During a data synchronization task, you can add objects to or remove objects from the task.