Two-way synchronization between Tair (Redis-compatible) - Data Transmission Service

Prerequisites

You must have a source instance and a destination Tair (Enterprise Edition) instance. For more information, see Step 1: Create an instance.
The source and destination instances should have the same storage capacity.

Limitations

Type	Description
Source database limitations	Two-way synchronization is supported only between Tair (Enterprise Edition) instances. Supported instances include: Redis open source edition: instances where Deployment Mode is set to Classic and Version Type is set to Tair (Enterprise Edition). Tair (Enterprise Edition): all instances. Currently, when you use Express Connect, VPN Gateway, or Smart Access Gateway to connect to Tair (Enterprise Edition), the two-way synchronization link feature is in limited availability. If you require this feature, please submit a ticket to contact us for assistance. DTS inserts a key prefixed with DTS_REDIS_TIMESTAMP_HEARTBEAT into the source database to record update timestamps. For clusters, this key is inserted into each shard. The key is filtered during synchronization and expires after the task completes. If the source database is a read-only instance or the DTS account does not have write (SETEX) permissions, the reported latency may be inaccurate. To ensure connection stability, increase the value of the `repl-backlog-size` parameter in the `redis.conf` file. Do not run `FLUSHDB` or `FLUSHALL` on the source database. This causes data inconsistency between the source and destination. If some keys in the source database have an expiration policy, they may not be deleted immediately upon expiry. The key count in the destination (viewable with `info`) may be lower than in the source. If the source instance is a Tair (Enterprise Edition) instance whose Storage Medium is Persistent Memory, ensure that the appendonly parameter is set to `yes` as described in Steps. If the source Redis instance uses extended modules such as RedisJSON, RediSearch, RedisBloom, RedisTimeSeries, RedisGraph, RedisAI, RedisGears, RedisCell, RediSQL, redis-tdigest, or RedisCompat, DTS cannot parse their RDB storage formats and the task fails. If the source is a child instance of a Tair (Enterprise Edition) Global Distributed Cache, DTS can synchronize data only from the selected child instance. Data from other child instances in the group is not synchronized.
Other limitations	Operations on the source or target Redis instance during synchronization, such as scaling, changing specifications, migrating availability zones, or changing the endpoint or port, can interrupt the task. DTS cannot obtain continuous log data or correct connection information. If this occurs, clear the synchronized data from the target before you reconfigure the task. If the source or destination is a self-managed Redis (where Access Method is not Alibaba Cloud Instance) and its endpoint changes during synchronization (for example, due to a migration or primary/standby switchover), the task may experience retries, latency, or failures, and data inconsistency may occur. Monitor the task status and reconfigure it if abnormalities persist. During an instance migration on a target Redis instance, such as a primary/standby switchover, data may be written only to memory and not replicated to the standby instance, which can lead to data loss. To ensure compatibility, use the same version for the source and destination databases or synchronize from an earlier version to a later one. During initial full data synchronization, DTS consumes read and write resources on the source and destination, which may increase load. Evaluate database performance beforehand and synchronize during off-peak hours. During DTS synchronization, do not write data to the destination database from any source other than DTS. Otherwise, data inconsistency between the source and destination databases will result. A two-way synchronization task includes forward and reverse synchronization tasks. When you configure or reset the task, if the destination object of one task matches the synchronization object of the other task: Allow only one task to synchronize full and incremental data. The other task supports only incremental synchronization. Data from the source of the current task synchronizes only to the destination of the current task. It does not serve as source data for the other task. During synchronization, if a key exists in both databases but has different values, the data with the later update time overwrites the data at the destination. If the destination instance is a cluster and one of its shards reaches its memory limit, or if the destination instance runs out of storage space, the DTS task fails due to an out-of-memory (OOM) error. If the target database runs out of memory and data eviction is triggered, data inconsistency occurs between the target and source because the default eviction policy (maxmemory-policy) of Tair (Redis OSS-Compatible) is volatile-lru. The task continues to run normally. To prevent this, set the eviction policy of the target database to noeviction. The task and writes fail when memory is exhausted, but no data is lost to eviction. Note Redis data eviction policies. DTS does not currently support synchronization if Transparent Data Encryption (TDE) is enabled on the source or destination instance. Full data may be re-synchronized to the destination and cause data inconsistency under the following conditions: A transient connection interruption occurs on the source or destination Redis instance, and resumable transmission fails. A primary/standby switchover or a failover occurs on the source or destination Redis. The endpoint of the source or destination Redis instance changes. The synchronized objects of the DTS instance are modified. If a Tair (Redis OSS-Compatible) instance has Transport Layer Security (TLS) encryption enabled, you must connect to it by using the SSL-encrypted method in DTS. TLSv1.3 is not supported. You cannot connect to an SSL-enabled Tair (Redis OSS-Compatible) instance by setting the Alibaba Cloud Instance to Alibaba Cloud Instance. Restarting a synchronization instance that includes both full and incremental tasks may cause DTS to rerun both. Restarting a synchronization instance may cause commands to execute more than once, leading to data inconsistency if non-idempotent commands such as `INCRBY` and `LPUSH` exist, or if full data re-synchronization is triggered. If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters. Note Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

Billing

Type	Fee
Full data synchronization	Free.
Incremental data synchronization	Charged. For details, see Billing overview.

Two-way synchronization topologies

Currently, DTS supports two-way synchronization between two Tair instances only. Synchronization among more than two Tair instances is not yet supported.

Supported Redis commands

APPEND
BITOP, BLPOP, BRPOP, BRPOPLPUSH
DECR, DECRBY, DEL
EVAL, EVALSHA, EXEC, EXPIRE, EXPIREAT
GEOADD, GETSET
HDEL, HINCRBY, HINCRBYFLOAT, HMSET, HSET, HSETNX
INCR, INCRBY, INCRBYFLOAT
LINSERT, LPOP, LPUSH, LPUSHX, LREM, LSET, LTRIM
MOVE, MSET, MSETNX, MULTI
PERSIST, PEXPIRE, PEXPIREAT, PFADD, PFMERGE, PSETEX
RENAME, RENAMENX, RESTORE, RPOP, RPOPLPUSH, RPUSH, RPUSHX
SADD, SDIFFSTORE, SELECT, SET, SETBIT, SETEX, SETNX, SETRANGE, SINTERSTORE, SMOVE, SPOP, SREM, SUNIONSTORE
ZADD, ZINCRBY, ZINTERSTORE, ZREM, ZREMRANGEBYLEX, ZUNIONSTORE, ZREMRANGEBYRANK, ZREMRANGEBYSCORE
XADD, XCLAIM, XDEL, XAUTOCLAIM, XGROUP CREATECONSUMER, XTRIM

Note

The PUBLISH command is not supported.
DTS cannot guarantee successful execution of Lua scripts called with EVAL or EVALSHA during incremental synchronization because the destination database does not return an explicit result.
For List data types, DTS does not clear existing data from the destination during sync or psync retransmissions, which may create duplicate data.

Database account permissions

Database	Permissions and authorization
source Tair instance	Both the source and destination Tair instances require database accounts with read and write permissions. For details, see Create and manage database accounts.
destination Tair instance

Procedure

Go to the data synchronization task list page in the destination region. You can do this in one of two ways.
DTS console
1. Log on to the DTS console.
2. In the navigation pane on the left, click Data Synchronization.
3. In the upper-left corner of the page, select the region where the synchronization instance is located.
DMS console
Note
The actual steps may vary depending on the mode and layout of the DMS console. For more information, see Simple mode console and Customize DMS console layout and style.
1. Log on to the DMS console.
2. In the top menu bar, choose Data + AI > DTS (DTS) > Data Synchronization.
3. To the right of Data Synchronization Tasks, select the region of the synchronization instance.
Click Create Task to open the task configuration page.

Configure the source and destination databases.

Warning

After you select the source and destination instances, review the Limits at the top of the page. Otherwise, the task may fail or data inconsistency may occur.

Category	Parameter	Description
N/A	Task Name	DTS automatically generates a task name. We recommend that you specify a descriptive name for easy identification. The name does not need to be unique.
Source Database	Select Existing Connection	Select the registered database instance with DTS from the drop-down list. The database information below is automatically configured. Note In the DMS console, this configuration item is Select a DMS database instance. If you have not registered the database instance or do not need to use a registered instance, manually configure the database information below.
	Database Type	Select Tair/Redis.
	Connection Type	Select Alibaba Cloud Instance.
	Instance Region	Select the region where the source Tair instance is located.
	Replicate Data Across Alibaba Cloud Accounts	This example synchronizes data within a single Alibaba Cloud account, so select No.
	Instance ID	Select the ID of the source Tair instance.
	Authentication Method	Select an authentication method based on your requirements. In this example, Password Login is selected. Note Only Redis 6.0 and later support Account + Password Login. If you select Secret-free login, make sure that password-free access is enabled for the Redis database. For information about how to enable password-free access for a Tair (Redis OSS-Compatible) instance, see Enable password-free access over a VPC.
	Database Password	Enter the password for the database account used to connect to the source Tair instance. For information about the required account permissions, see Permissions required for database accounts. Note The database password must be in the <user>:<password> format. For example, if the custom username for the Tair instance is `admin` and the password is `Rp829dlwa`, enter `admin:Rp829dlwa`.
	Encryption	Select Non-encrypted or SSL-encrypted based on your database instance requirements. Note If you are using a self-managed Redis instance (the Access Method is not Alibaba Cloud Instance) and you select SSL-encrypted, you must also upload a CA Certificate and enter the CA Certificate Password.
Destination Database	Select Existing Connection	Select the registered database instance with DTS from the drop-down list. The database information below is automatically configured. Note In the DMS console, this configuration item is Select a DMS database instance. If you have not registered the database instance or do not need to use a registered instance, manually configure the database information below.
	Database Type	Select Tair/Redis.
	Connection Type	Select Alibaba Cloud Instance.
	Instance Region	Select the region where the destination Tair instance is located.
	Replicate Data Across Alibaba Cloud Accounts	This example synchronizes data within a single Alibaba Cloud account, so select No.
	Instance ID	Select the ID of the destination Tair instance.
	Authentication Method	Select an authentication method based on your requirements. In this example, Password Login is selected. Note Only Redis 6.0 and later support Account + Password Login. If you select Secret-free login, make sure that password-free access is enabled for the Redis database. For information about how to enable password-free access for a Tair (Redis OSS-Compatible) instance, see Enable password-free access over a VPC.
	Database Password	Enter the password for the database account used to connect to the destination Tair instance. For information about the required account permissions, see Permissions required for database accounts. Note If the custom username for the Tair instance is `admin` and the password is `Rp829dlwa`, enter `admin:Rp829dlwa`.
	Encryption	Select Non-encrypted or SSL-encrypted based on your database instance requirements. Note If you are using a self-managed Redis instance (the Access Method is not Alibaba Cloud Instance) and you select SSL-encrypted, you must also upload a CA Certificate and enter the CA Certificate Password.

After you complete the configuration, click Test Connectivity and Proceed at the bottom of the page.
Note
Ensure that the IP address blocks of the DTS service are added to the security settings of the source and destination databases, either automatically or manually, to allow access from DTS servers. For more information, see Add the IP address whitelist of DTS servers.

Configure the task objects.

On the Configure Objects page, specify the objects to synchronize.

Parameter	Description
Synchronization Types	By default, Incremental Data Synchronization is selected. You must also select Full Data Synchronization. After the precheck is complete, DTS performs a full data synchronization from the source instance to the destination cluster. This initial full sync creates a baseline for subsequent incremental synchronization.
Synchronization Topology	Select Two-way Synchronization.
Processing Mode of Conflicting Tables	Precheck and Report Errors: Checks if the destination is empty. If the destination is empty, the precheck is successful. If the destination is not empty, an error is reported during the precheck, and the data synchronization job does not start. Ignore Errors and Proceed: Skips the Check the existence of objects in the destination database. precheck. Warning If you select Ignore Errors and Proceed, source data overwrites destination data if the same key exists in both the source and destination. This may cause data loss in the destination. Select this option with caution.
Source Objects	In the Source Objects box, click the objects, and then click to move them to the Selected Objects box. Note You can select objects only at the database level. To synchronize data for specific keys within a database, use the data filtering feature in the Selected Objects pane.
Selected Objects	To specify a destination database (DB 0 to DB 255) or filter data by prefix, use the mapping or filtering features. In the Selected Objects pane, right-click the database to synchronize, and then configure the settings in the Edit Schema dialog box that appears. For more information, see Map object names and Specify filter conditions.

Click Next: Advanced Settings.

Parameter	Description
Dedicated Cluster for Task Scheduling	By default, DTS uses a shared cluster for tasks, so you do not need to make a selection. For greater task stability, you can purchase a dedicated cluster to run the DTS synchronization task. For more information, see What is a DTS dedicated cluster?.
Retry Time for Failed Connections	If the connection to the source or destination database fails after the synchronization task starts, DTS reports an error and immediately begins to retry the connection. The default retry duration is 720 minutes. You can customize the retry time to a value from 10 to 1,440 minutes. We recommend a duration of 30 minutes or more. If the connection is restored within this period, the task resumes automatically. Otherwise, the task fails. Note If multiple DTS instances (e.g., Instance A and B) share a source or destination, DTS uses the shortest configured retry duration (e.g., 30 minutes for A, 60 for B, so 30 minutes is used) for all instances. DTS charges for task runtime during connection retries. Set a custom duration based on your business needs, or release the DTS instance promptly after you release the source/destination instances.
Retry Time for Other Issues	If a non-connection issue (e.g., a DDL or DML execution error) occurs, DTS reports an error and immediately retries the operation. The default retry duration is 10 minutes. You can also customize the retry time to a value from 1 to 1,440 minutes. We recommend a duration of 10 minutes or more. If the related operations succeed within the set retry time, the synchronization task automatically resumes. Otherwise, the task fails. Important The value of Retry Time for Other Issues must be less than that of Retry Time for Failed Connections.
Enable Throttling for Full Data Synchronization	During full data synchronization, DTS consumes read and write resources from the source and destination databases, which can increase their load. To mitigate pressure on the destination database, you can limit the migration rate by setting Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s). Note This parameter is available only when Synchronization Types is selected for Full Data Synchronization.
Enable Throttling for Incremental Data Synchronization	You can also limit the incremental synchronization rate to reduce pressure on the destination database by setting RPS of Incremental Data Synchronization and Data synchronization speed for incremental synchronization (MB/s).
Extend Expiration Time of Destination Database Key	Specifies the extra time by which to extend the TTL of keys when they are migrated from the source database to the destination database. To ensure data consistency, we recommend that you extend the key expiration time if commands such as the following are used. Note In scenarios that involve distributed locks, this may prevent the locks from being released in a timely manner. `EXPIRE key seconds PEXPIRE key milliseconds EXPIREAT key timestamp PEXPIREAT key timestampMs`
Environment Tag	You can select an environment tag to identify the instance based on your business requirements. In this example, you do not need to select a tag.
Configure ETL	Choose whether to enable the extract, transform, and load (ETL) feature. For more information, see What is ETL? Valid values: Yes: Enables the ETL feature. Enter data processing statements in the code editor. For more information, see Configure ETL in a data migration or data synchronization task. No: Disables the ETL feature.
Monitoring and Alerting	Choose whether to set up alerts. If the synchronization fails or the latency exceeds the specified threshold, DTS sends a notification to the alert contacts. No: No alerts are configured. Yes: Configures alerts. You must also set the alert threshold and alert notifications. For more information, see Configure monitoring and alerting during task configuration.

Click Data Verification to configure a data verification task.
To use the data verification feature, see Configure data verification.

Save the task and perform a precheck.
- To view the parameters for configuring this instance via an API operation, hover over the Next: Save Task Settings and Precheck button and click Preview OpenAPI parameters in the tooltip.
- If you have finished viewing the API parameters, click Next: Save Task Settings and Precheck at the bottom of the page.
Note
- Before a synchronization task starts, DTS performs a precheck. You can start the task only if the precheck passes.
- If the precheck fails, click View Details next to the failed item, fix the issue as prompted, and then rerun the precheck.
- If the precheck generates warnings:
  For non-ignorable warning, click View Details next to the item, fix the issue as prompted, and run the precheck again.
  For ignorable warnings, you can bypass them by clicking Confirm Alert Details, then Ignore, and then OK. Finally, click Precheck Again to skip the warning and run the precheck again. Ignoring precheck warnings may lead to data inconsistencies and other business risks. Proceed with caution.

Purchase the instance.

When the Success Rate reaches 100%, click Next: Purchase Instance.

On the Purchase page, select the billing method and link specifications for the data synchronization instance. For more information, see the following table.

Category	Parameter	Description
New Instance Class	Billing Method	Subscription: You pay upfront for a specific duration. This is cost-effective for long-term, continuous tasks. Pay-as-you-go: You are billed hourly for actual usage. This is ideal for short-term or test tasks, as you can release the instance at any time to save costs.
	Resource Group Settings	The resource group to which the instance belongs. The default is default resource group. For more information, see What is Resource Management?.
	Instance Class	DTS offers synchronization specifications at different performance levels that affect the synchronization rate. Select a specification based on your business requirements. For more information, see Data synchronization link specifications.
	Subscription Duration	In subscription mode, select the duration and quantity of the instance. Monthly options range from 1 to 9 months. Yearly options include 1, 2, 3, or 5 years. Note This option appears only when the billing method is Subscription.

Read and select the checkbox for Data Transmission Service (Pay-as-you-go) Service Terms.
Click Buy and Start, and then click OK in the OK dialog box.
You can monitor the task progress on the data synchronization page.

Configure the reverse synchronization task.
1. Wait for the forward synchronization task to finish its initialization. The task is ready when its Status changes to Running.
  
  Note
  If your DTS instance is configured for both full and incremental data synchronization (meaning Synchronization Types includes both Full Data Synchronization and Incremental Data Synchronization), the task appears as Incremental Data Synchronization in the synchronization task list.
2. Locate the reverse synchronization task and click Configure Task.
3. Configure the reverse synchronization task by following Step 3 to Step 6.
  Important
  
  When configuring the reverse synchronization task, ensure that you select the correct instances: the source instance for the reverse task is the destination instance from the forward task, and vice versa. You must also carefully confirm that the instance information, such as database names, accounts, and passwords, is consistent.
  
  You cannot modify the Instance Region for the source and destination instances of the reverse synchronization task. Fewer parameters need to be configured for a reverse synchronization task than for a forward one. Configure the parameters as prompted on the console.
  
  The Processing Mode of Conflicting Tables setting for the reverse synchronization task does not check for tables that are synchronized to the destination instance by the forward synchronization task.
  
  The reverse synchronization task does not support synchronizing the objects selected in the Selected Objects box of the forward synchronization task.
  
  Do not use the mapping feature when you configure the reverse task, as it may cause data inconsistency.
4. When the Success Rate reaches 100%, click Back.
After you configure the reverse synchronization task, wait until the Status of both synchronization tasks is Running. The two-way data synchronization setup is now complete.