Data Transmission Service:Two-way synchronization between Tair (Redis OSS-Compatible) Enhanced Edition instances

Prerequisites

Precautions

• For two-way synchronization, the forward synchronization task performs both full and incremental data synchronization. The reverse synchronization task performs only incremental data synchronization.

  Warning

  To prevent data inconsistency, do not modify or write to the same key in both instances while the two-way synchronization task is running.

• During full data synchronization, DTS consumes resources from both the source and destination instances, which can increase the server load. If your database has a heavy workload or low specifications, the increased load might impact performance or even cause service interruptions. We recommend that you evaluate the performance impact before starting the task and run the synchronization task during off-peak hours.

• Do not run the FLUSHDB or FLUSHALL command on the source instance during data synchronization. Doing so will cause data inconsistency between the source and destination instances.

• If the destination database runs out of memory and triggers data eviction, data may become inconsistent between the source and destination. This occurs because the default data eviction policy (maxmemory-policy) for Tair (Redis OSS-Compatible) is volatile-lru. However, this does not affect normal task operation.

  To prevent this issue, we recommend setting the data eviction policy for the destination database to noeviction. With this policy, if the destination runs out of memory, write operations fail, the task stops, and no data is lost to eviction.

  Note

  For more information about data eviction policies, see Data eviction policies in Redis.

• If the source database uses expiration policies for some keys, the destination database might appear to have fewer keys (as reported by commands like info). This can happen because expired keys may not be deleted immediately in the source database.

  Note

  The number of keys that do not have an expiration policy or have not expired is the same in both the source and destination databases.

• If the destination Redis instance does not have direct connection enabled, DTS uses a proxy to write data to the instance.

  Note

  For more information about how to enable direct connection, see Enable direct connection.

• If you scale the source or destination instance (for example, by adding or removing shards) or change specifications (such as increasing memory) during synchronization, you must reconfigure the task. To ensure data consistency, we recommend that you clear all synchronized data from both instances before you reconfigure the task.

• If the connection endpoint of the source or destination Redis instance changes during synchronization, such as from a zone migration or a network switch from classic network to Virtual Private Cloud (VPC), you must reconfigure the task.

• The following limitations apply when you synchronize data from a standalone Redis instance to a cluster-mode Redis instance. A Redis cluster only allows a command to operate on keys that are in the same slot. If you perform a multi-key operation on the source database where the keys span multiple slots, the following error occurs:

  CROSSSLOT Keys in request don't hash to the same slot

  We recommend that you perform only single-key operations during data synchronization to prevent task interruptions.

• If the destination instance uses the cluster architecture, the DTS task fails with an Out of Memory error if any shard reaches its memory limit or if the instance runs out of storage space.

• If transparent data encryption (TDE) is enabled on the source or destination instance, you cannot use DTS to synchronize data.

• A two-way synchronization task includes forward and reverse synchronization tasks. When you configure or reset the task, if the destination object of one task matches the synchronization object of the other task:

  • Allow only one task to synchronize full and incremental data. The other task supports only incremental synchronization.

  • Data from the source of the current task synchronizes only to the destination of the current task. It does not serve as source data for the other task.

Billing

Supported synchronization commands

• APPEND

• BITOP, BLPOP, BRPOP, and BRPOPLPUSH

• DECR, DECRBY, and DEL

• EVAL, EVALSHA, EXEC, EXPIRE, and EXPIREAT

• GEOADD and GETSET

• HDEL, HINCRBY, HINCRBYFLOAT, HMSET, HSET, and HSETNX

• INCR, INCRBY, and INCRBYFLOAT

• LINSERT, LPOP, LPUSH, LPUSHX, LREM, LSET, and LTRIM

• MOVE, MSET, MSETNX, and MULTI

• PERSIST, PEXPIRE, PEXPIREAT, PFADD, PFMERGE, and PSETEX

• RENAME, RENAMENX, RPOP, RPOPLPUSH, RPUSH, and RPUSHX

• SADD, SDIFFSTORE, SELECT, SET, SETBIT, SETEX, SETNX, SETRANGE, SINTERSTORE, SMOVE, SPOP, SREM, and SUNIONSTORE

• UNLINK, ZADD, ZINCRBY, ZINTERSTORE, ZREM, ZREMRANGEBYLEX, ZUNIONSTORE, ZREMRANGEBYRANK, and ZREMRANGEBYSCORE

• SWAPDB (This command is not supported if the source or destination instance uses the cluster architecture.)

Database account permissions

Procedure

1. Purchase a DTS instance. For more information, see Purchase guide.

   Important

   On the purchase page, set the source instance type to Redis, the destination instance type to Redis, and the synchronization topology to Two-way Synchronization.

2. Log in to the Data Transmission Service (DTS) console.

   Note

   If you are automatically redirected to the Data Management (DMS) console, you can click the icon in the lower-right corner and then click to return to the classic DTS console.

3. In the left-side navigation pane, click Data Synchronization.

4. At the top of the Synchronization Tasks page, select the region of your destination instance.

5. Configure the forward synchronization task.

   1. Find the data synchronization instance that you purchased. In the Actions column for the first synchronization task, click Configure Synchronization Task.

      Important

      A two-way data synchronization instance contains two synchronization tasks, both of which must be configured separately. To configure the reverse synchronization task, find the second task and click Actions in the Configure Synchronization Task column.

      

   2. Configure the source and destination instances.

      Section	Parameter	Description
      N/A	Synchronization task name	DTS automatically generates a task name. For easier identification, we recommend giving the task a descriptive name. Names do not need to be unique.
      Source instance details	Instance type	Select Redis Instance.
      	Instance region	The region of the source instance that you selected when you purchased the DTS instance. This parameter cannot be changed.
      	Instance ID	Select the ID of the source Redis instance. Important When you configure the reverse synchronization task, select the ID of the destination instance that you configured for the forward task.
      	Database password	Enter the password for the Redis instance's database account. For information about account permissions, see Database account permissions. Important The password must be in the <user>:<password> format. For example, if the username is admin and the password is Rp829dlwa, enter admin:Rp829dlwa.
      Destination instance details	Instance type	Select Redis Instance.
      	Instance region	The region of the destination instance that you selected when you purchased the DTS instance. This parameter cannot be changed.
      	Instance ID	Select the ID of the destination Redis instance. Important When you configure the reverse synchronization task, select the ID of the source instance that you configured for the forward task.
      	Database password	Enter the password for the Redis instance's database account. For information about account permissions, see Database account permissions. Important The password must be in the <user>:<password> format. For example, if the username is admin and the password is Rp829dlwa, enter admin:Rp829dlwa.

   3. At the bottom of the page, click Set Whitelist and Next.

      Note

      • If the source or destination instance is an Alibaba Cloud database instance (such as ApsaraDB RDS for MySQL or ApsaraDB for MongoDB) or a self-managed database on an ECS instance, DTS automatically adds the CIDR blocks of DTS servers to the instance's whitelist or the security group rules of the ECS instance. You do not need to add them manually. For more information, see CIDR blocks of DTS servers.

      • After the DTS task is complete or released, we recommend that you manually remove the DTS server CIDR blocks.

   4. Configure the synchronization policy and objects.

      Section	Parameter	Description
      Synchronization policy	Conflict resolution policy	Overwrite: When a conflict occurs, DTS overwrites the conflicting record in the destination instance. If a conflict occurs because data records have the same key but different values, the record with the later update time overwrites the other record.
      	Processing mode in existed target table	Precheck and Report Errors: Checks whether the destination instance is empty. If it is empty, the precheck passes. If not, the task is stopped and an error is reported. Ignore Errors and Proceed: Skips the check for an empty destination database. Warning If you select Ignore Errors and Proceed, data from the source instance overwrites any data in the destination instance that has the same key during initialization. Use this option with caution.
      Select synchronization objects	N/A	In the Source Objects box, click the database that you want to synchronize, and then click the You can only synchronize entire databases, not individual keys.
      Rename databases and tables	N/A	This feature is not supported.
      Specify whether to copy temporary tables to the destination database when DMS performs online DDL operations on the source tables	N/A	If the source database uses Data Management (DMS) to execute an Online DDL change, you can choose whether to synchronize the temporary table data generated by the Online DDL change. Yes: Synchronizes the data from temporary tables generated by Online DDL operations. Note If a large amount of data is generated in temporary tables during Online DDL operations, the data synchronization task may be delayed. No: Does not synchronize data from temporary tables. Only the original DDL data from the source database is synchronized. Note This option may cause table locks on the destination database.
      Configure retry time for failed connections	N/A	If DTS cannot connect to the source or destination database, it retries the connection for 720 minutes (12 hours) by default. You can also specify a custom retry duration. If DTS reconnects to the databases within the specified duration, the data synchronization task automatically resumes. Otherwise, the task fails. Note You are charged for the DTS instance during the connection retry period. We recommend that you specify a custom retry duration based on your business requirements or release the DTS instance promptly after the source and destination database instances are released.

   5. After you complete the configurations, click Next in the lower-right corner of the page.

   6. Configure initialization options.

      The synchronization type is fixed as Include full data + incremental data. DTS first performs a full data synchronization from the source to the destination instance, and then synchronizes incremental data.

      Important

      • When you configure the reverse synchronization task, if the objects have already been initialized in the destination instance, only incremental data is synchronized.

      • If a version-related error message appears, upgrade the source Redis instance to the specified version as prompted. For more information, see Upgrade the major version and Upgrade the minor version and proxy version.

   7. After you complete the configuration, click Precheck and Start in the lower-right corner of the page.

      Note

      • DTS performs a precheck before the synchronization task starts. The task can begin only after it passes this check.

      • If the precheck fails, click the icon next to the failed item to view details.

        • Fix the issues based on the cause and run the precheck again.

        • If you do not need to fix an alert item, you can click Ignore or Ignore and Re-precheck to skip the alert item and run the precheck again.

   8. After the Precheck dialog box displays Precheck Passed, close the Precheck dialog box to start the forward synchronization task.

6. Wait for the forward task to complete initialization and enter the Synchronizing state.

   You can view the status of the data synchronization task on the Data Synchronization page.

7. Configure the reverse synchronization task.

   1. Find the second synchronization task and click Configure Synchronization Channel.

   2. Repeat the substeps in Step 5 to configure the reverse synchronization task.