Tair (Redis® OSS-Compatible):Two-way synchronization for Tair (Enterprise Edition)

You can configure the task in the console or by using the OpenAPI. For more information, see Configure one-way or two-way data synchronization between Tair (Enterprise Edition) instances by calling OpenAPI.

Overview

Two-way synchronization requires two synchronization links: a forward task and a reverse task. The forward task performs both full data migration and incremental data synchronization. The reverse task performs only incremental data synchronization.

• Full data migration: DTS migrates all existing data from the source database to the destination database. Full data migration is free of charge.

• Incremental data synchronization: After the full data migration, DTS synchronizes incremental data changes from the source database to the destination database in real time. You are charged for incremental data synchronization based on its duration, not the amount of data transferred. For more information, see Billable items.

Prerequisites

Precautions

• Do not change the source or destination database configuration during migration. Scaling resources, changing specifications, or modifying connection endpoints causes the task to fail, requiring reconfiguration. Because the migration consumes resources on both databases, perform it during off-peak hours.

• For instances outside the Chinese mainland, two-way synchronization is supported only between instances in the same region. For example, two instances in the Japan (Tokyo) region can be synchronized, but not an instance in Japan (Tokyo) and one in Germany (Frankfurt).

• A two-way synchronization task includes forward and reverse synchronization tasks. When you configure or reset the task, if the destination object of one task matches the synchronization object of the other task:

  • Allow only one task to synchronize full and incremental data. The other task supports only incremental synchronization.

  • Data from the source of the current task synchronizes only to the destination of the current task. It does not serve as source data for the other task.

Procedure

1. Navigate to the data synchronization task list.

   1. Log on to the Data Management (DMS) console.

   2. In the top navigation bar, choose Data + AI > Data Transmission (DTS) > Data Synchronization.

   3. Next to Data Synchronization Tasks, select the region of the instance that you want to synchronize.

2. Click Create Task.

3. Configure the source and destination databases, and then click Test Connectivity and Proceed at the bottom of the page.

   Section	Parameter	Description
   N/A	Task Name	DTS automatically generates a task name. We recommend that you specify a descriptive name for easy identification. The name does not need to be unique.
   Source Database	Select a DMS database instance.	If the source database is registered with DMS, select it here to auto-fill its details. Otherwise, skip this option.
   	Database Type	Select Tair/Redis.
   	Connection Type	Select Alibaba Cloud instance.
   	Instance Region	Select the region where the instance is located.
   	Replicate Data Across Alibaba Cloud Accounts	Select No, as this example covers migration within a single Alibaba Cloud account.
   	Instance ID	Select the ID of the source instance.
   	Authentication Method	Select Password Login or Secret-free login based on your requirements. In this example, Password Login is selected. Note If the password-free access feature is not enabled for the instance, select Password Login.
   	Database Password	Enter the password for the account used to connect to the source instance. Note This parameter is optional. If no password is set, you can leave it blank. If you use a custom account, it must have read permissions. The account and password must be in the format of <user>:<password>. For example, if the custom account is admin and the password is Rp829dlwa, enter admin:Rp829dlwa.
   Destination Database	Select a DMS database instance.	If the destination database is registered with DMS, select it here to auto-fill its details. Otherwise, skip this option.
   	Database Type	Tair/Redis is selected by default.
   	Connection Type	Select Alibaba Cloud instance.
   	Instance Region	Select the region where the destination instance is located.
   	Instance ID	Select the ID of the destination instance.
   	Authentication Method	Select Password Login or Secret-free login based on your requirements. In this example, Password Login is selected. Note If the password-free access feature is not enabled for the instance, select Password Login.
   	Database Password	Enter the password used to connect to the destination database. Note If you use a custom account, it must have write permissions. The account and password must be in the format of <user>:<password>. For example, if the custom account is admin and the password is Rp829dlwa, enter admin:Rp829dlwa.

4. Configure the task objects, and then click Next: Advanced Settings at the bottom of the page.

   Parameter	Description
   Synchronization Type	Incremental data synchronization is always enabled. You can optionally enable Full data synchronization.
   Synchronization Topology	Select Two-way Synchronization.
   Processing Mode for Existing Destination Tables	Precheck and Report Errors (default): Checks whether data (keys) exist in the destination database. During the precheck, the task fails if the destination database contains conflicting keys. The task proceeds if no such keys exist. Ignore Errors and Proceed: Skips the Check the existence of objects in the destination database. item. If a key with the same name already exists in the destination database, it is overwritten.
   Source Objects and Selected Objects	In the Source Objects box, select the objects to be synchronized, and then click to move them to the Selected Objects box. To remove a selected object, click it in the Selected Objects box, and then click to move it back to the Source Objects box. Note You can select objects at the database level (DB 0 to DB 255).

5. Configure the advanced settings, and then click Next: Data Verification.

   In most cases, you can use the default settings. For more information, see Appendix: Advanced settings.

6. Configure data verification, and then click Next: Save Task Settings and Precheck.

   In most cases, you can keep the default settings. For more information, see Configure data verification in a DTS synchronization or migration instance.

7. After the precheck is complete, click Next: Purchase Instance.

   If any Warning or Failed items are reported, resolve them individually. You can click View Details to see the issue and resolve it based on the provided suggestion. You can also click Confirm Alert Details to ignore the check item, but this is not recommended as it may cause data inconsistencies. For more information, see Precheck issues. After you resolve the issues, run the precheck again.

8. On the Purchase page, configure the required information and click Buy and Start.

   • (Optional) For Resource Group Settings, select the resource group for the DTS task. The default resource group is used by default.

   • (Optional) Select the specification of the DTS synchronization link. Higher specifications provide faster synchronization speeds but incur higher costs. The default value is large. For more information, see Specifications of data synchronization links.

   • Read and agree to the terms of service.

   After you complete the purchase, the synchronization task starts. You can view its progress on the Data Synchronization page.

9. Wait for the status of the forward synchronization task to become Running, and then click Configure Task for the reverse task.

10. Follow the steps above to configure the reverse task.

    When the Success Rate of the precheck reaches 100%, the configuration is complete. You can then click Back to List.

11. In the data synchronization list, when the status of both the forward and reverse tasks is Running, the two-way data synchronization setup is complete.

FAQ

• Why does the connectivity test fail?

  Check the following:

  • Invalid account or password. The Redis password format is user:password. For more information, see Connection methods for instances.

  • If the source is a self-managed database in an on-premises data center or on another cloud, a firewall may block access. To permit access, you must add the IP addresses of the DTS servers for the corresponding region to your allowlist. For more information, see Add the CIDR blocks of DTS servers to the allowlist of a database.

• Why does the migration task fail?

  • The migration task fails if you change the configuration (such as resources, specifications, or connection endpoints) of the source or destination database. If this occurs, you must reconfigure the task.

  • If the destination instance has insufficient memory, or if it is a cluster instance and a shard has reached its memory limit, the DTS task will fail with an out of memory (OOM) error.

  • If Transparent Data Encryption (TDE) is enabled on the destination instance, data migration with DTS is not supported.

• Why is there a data discrepancy?

  • If some keys in the source database use an expiration (Expire) policy, they may not be deleted immediately upon expiration, resulting in the destination database having fewer keys than the source database.

  • For List data types, DTS does not perform a FLUSH operation on existing data in the destination when transferring data with PSYNC or SYNC. This can lead to duplicate data.

  • If a network interruption occurs during full data migration, DTS may perform multiple full migrations, which automatically overwrite keys with the same name. If a delete operation is performed on the source database during this time, the command will not be synchronized to the destination, resulting in the destination database having more data than the source.

• Why does the precheck verify if the Redis eviction policy is noeviction?

  The default eviction policy (maxmemory-policy) for a Tair (Redis OSS-compatible) instance is volatile-lru. If the destination runs out of memory, eviction is triggered, causing data inconsistency without failing the task. To prevent this, set the destination eviction policy to noeviction. With this policy, writes fail when memory is full, causing the migration task to fail but preventing data loss from eviction. For more information, see Redis data eviction policies.

• Why do I receive the CROSSSLOT Keys in request don't hash to the same slot error?

  This error occurs because the cluster architecture of Redis does not support cross-slot operations in a single command. To prevent the link from being interrupted, perform only single-key operations during DTS synchronization.

• Which commands are supported for synchronization?

  • The following commands are supported:

    • APPEND

    • BITOP, BLPOP, BRPOP, and BRPOPLPUSH

    • DECR, DECRBY, and DEL

    • EVAL, EVALSHA, EXEC, EXPIRE, and EXPIREAT

    • GEOADD and GETSET

    • HDEL, HINCRBY, HINCRBYFLOAT, HMSET, HSET, and HSETNX

    • INCR, INCRBY, and INCRBYFLOAT

    • LINSERT, LPOP, LPUSH, LPUSHX, LREM, LSET, and LTRIM

    • MOVE, MSET, MSETNX, and MULTI

    • PERSIST, PEXPIRE, PEXPIREAT, PFADD, PFMERGE, and PSETEX

    • RENAME, RENAMENX, RESTORE, RPOP, RPOPLPUSH, RPUSH, and RPUSHX

    • SADD, SDIFFSTORE, SELECT, SET, SETBIT, SETEX, SETNX, SETRANGE, SINTERSTORE, SMOVE, SPOP, SREM, and SUNIONSTORE

    • ZADD, ZINCRBY, ZINTERSTORE, ZREM, ZREMRANGEBYLEX, ZUNIONSTORE, ZREMRANGEBYRANK, and ZREMRANGEBYSCORE

    • SWAPDB and UNLINK (supported only when the source Redis instance is version 4.0)

    • XADD, XCLAIM, XDEL, XAUTOCLAIM, XGROUP CREATECONSUMER, and XTRIM

  • The PUBLISH command is not supported.

  • DTS cannot guarantee the successful execution of a Lua script called with EVAL or EVALSHA during incremental data synchronization because the destination does not explicitly return an execution result.