Tair (Redis® OSS-Compatible):Migrate from an on-premises or third-party cloud database to Alibaba Cloud Tair

Overview

• Full data migration

  DTS migrates all existing data from the source database to the destination database. Full data migration is free of charge.

• Incremental data migration

  After full data migration completes, DTS synchronizes data changes from the source to the destination in real time. Incremental migration requires the ability to run the PSYNC or SYNC command on the source database. Otherwise, only full data migration is available. Incremental migration is billed by duration, not by data volume. For more information, see Billable items.

  Note

  To ensure successful incremental data migration, disable the replication output buffer limit on the source database by running the following command: CONFIG SET client-output-buffer-limit 'slave 0 0 0'.

Prerequisites

A Tair (Redis OSS-compatible) instance is created with more memory than the self-managed Redis database uses. For more information, see Create an instance.

Usage notes

Do not change the source or destination database configuration during migration. Scaling resources, changing specifications, or modifying connection endpoints causes the task to fail, requiring reconfiguration. Because the migration consumes resources on both databases, perform it during off-peak hours.

Procedure

1. Go to the Data Migration Tasks page.

   1. Log on to the Data Management (DMS) console.

   2. In the top navigation bar, choose Data+AI > Data Transmission (DTS) > Data Migration.

   3. On the right side of Data Migration Tasks, select the region where your destination instance is located.

2. Click Create Task.

3. Configure the source and destination databases, and then click Test Connectivity and Proceed.

   Section	Parameter	Description
   N/A	Task Name	DTS automatically generates a task name. We recommend that you specify a descriptive name for easy identification. The name does not need to be unique.
   Source Database	Select a DMS database instance.	If the source database is registered with DMS, select it from the drop-down list to auto-fill connection details. Otherwise, skip this option.
   	Database Type	Select Tair/Redis.
   	Connection Type	Select a connection type based on where your source database is deployed. If your instance is in an on-premises data center or on another cloud, select Public IP Address. This example uses Self-managed Database on ECS.
   	Instance Region	Select the region where the ECS instance is located. If the instance is in an on-premises data center or on another cloud, select the region closest to the source database.
   	Replicate Data Across Alibaba Cloud Accounts	This example shows migration within the same Alibaba Cloud account. Select No.
   	ECS Instance ID	Select the ID of the ECS instance where the source database is located. Note If the source database is a cluster, select the ECS instance ID of any master node.
   	Instance Mode	Select the architecture of your source database: Standalone: Standard architecture (master-replica). Cluster: Cluster architecture. If your source database is a cluster, enter the port number of any master node in the Port parameter.
   	Port	Enter the port of the source Redis database. The default port is 6379.
   	Authentication Method	Select Password Login or Logon without password. Note If your self-managed Redis database does not have a password, you can select Logon without password.
   	Database Password	Enter the password to connect to the source Redis database. Note This parameter is optional. If no password is set, you can leave this field empty. If you use a custom account, it must have read permissions. The account and password must be in the format of <user>:<password>. For example, if the custom account for the Redis instance is admin and the password is Rp829dlwa, enter admin:Rp829dlwa as the database password.
   	Encryption	Select Non-encrypted or SSL-encrypted. Note If your self-managed Redis (where the Access Method is not Cloud instance) uses an SSL-encrypted, you must also upload a CA Certificate and enter the CA Key.
   Destination Database	Select a DMS database instance.	If the destination database is registered with DMS, select it from the drop-down list to auto-fill connection details. Otherwise, skip this option.
   	Database Type	Tair/Redis is selected by default.
   	Connection Type	Select Cloud instance.
   	Instance Region	Select the region of the destination instance.
   	Instance ID	Select the ID of the destination instance.
   	Authentication Method	Select Password Login or Logon without password. This example uses Password Login. Note If the VPC password-free access feature is not enabled for the Tair (Redis OSS-compatible) instance, select Password Login.
   	Database Password	Enter the password to connect to the destination database. Note If you use a custom account, it must have write permissions. The account and password must be in the format of <user>:<password>. For example, if the custom account for the Redis instance is admin and the password is Rp829dlwa, enter admin:Rp829dlwa as the database password.
   	Encryption	Select Non-encrypted or SSL-encrypted.

4. Configure migration objects, and then click Next: Advanced Settings.

   Parameter	Description
   Migration Types	Select a Migration Types. Full + Incremental (Default): Uses Redis native synchronization to write data as a memory snapshot to the destination, enabling migration with minimal downtime. If you do not have SYNC or PSYNC permissions on the source database, select Full Data Migration. Full Data Migration: Uses the SCAN command to iterate through the entire source and write the data to the destination. To ensure data consistency, avoid writing new data to the source instance during the migration.
   Processing Mode for Existing Destination Tables	Precheck and Report Errors (Default): Checks if data (keys) exist in the destination database. If keys exist, the precheck reports an error and the migration task does not start. If no keys exist, the precheck passes. Ignore Errors and Proceed: Skips the Check the existence of objects in the destination database. item. If a Key with the same name already exists in the target database, it will be overwritten.
   Source Objects and Selected Objects	In the Source Objects box, select the objects to migrate and click the right arrow to move them to the Selected Objects box. To remove an object, select it in the Selected Objects box and click the left arrow to move it back to the Source Objects box. Note Migration objects are selected at the database level (DB 0 to DB 255).

5. Configure advanced settings, and then click Next: Data Verification.

   In most cases, you can keep the default settings. For more information, see Appendix: Advanced settings.

6. Configure data verification, and then click Next: Save Task Settings and Precheck.

   In most cases, you can keep the default settings. For more information, see Configure data verification in a DTS synchronization or migration instance.

7. After the precheck is complete, click Next: Purchase Instance.

   If any Warning or Failed items are reported, resolve them individually. You can click View Details to see the issue and resolve it based on the provided suggestion. You can also click Confirm Alert Details to ignore the check item, but this is not recommended as it may cause data inconsistencies. For more information, see Precheck issues. After you resolve the issues, run the precheck again.

8. On the Purchase page, configure the settings and click Buy and Start.

   • (Optional) Select the Resource Group Settings for the DTS migration link. The default is default resource group.

   • (Optional) Select the specifications for the DTS migration link. Higher specifications offer faster migration speeds but incur higher costs. The default is large. For more information, see Specifications of data migration links.

   • Read and agree to the terms of service.

   After the purchase, the migration task starts. You can view its progress on the Data Migration page.

Next steps

• If you used incremental data migration, you must manually end or release the task in the console after the migration is complete.

• You can verify the migrated data. For more information, see Verify migrated data.

Related documents

If online migration is not required, you can use redis-cli to import an AOF file instead. For more information, see Migrate data from an AOF file.

FAQ

• Why does the connectivity test fail?

  Check the following:

  • Invalid account or password. The Redis password format is user:password. For more information, see Connection methods for instances.

  • If the source is a self-managed database in an on-premises data center or on another cloud, a firewall may block access. To permit access, you must add the IP addresses of the DTS servers for the corresponding region to your allowlist. For more information, see Add the CIDR blocks of DTS servers to the allowlist of a database.

• Why does the migration task fail?

  • The migration task fails if you change the configuration (such as resources, specifications, or connection endpoints) of the source or destination database. If this occurs, you must reconfigure the task.

  • If the destination instance has insufficient memory, or if it is a cluster instance and a shard has reached its memory limit, the DTS task will fail with an out of memory (OOM) error.

  • If Transparent Data Encryption (TDE) is enabled on the destination instance, data migration with DTS is not supported.

• Why is there a data discrepancy?

  • If some keys in the source database use an expiration (Expire) policy, they may not be deleted immediately upon expiration, resulting in the destination database having fewer keys than the source database.

  • For List data types, DTS does not perform a FLUSH operation on existing data in the destination when transferring data with PSYNC or SYNC. This can lead to duplicate data.

  • If a network interruption occurs during full data migration, DTS may perform multiple full migrations, which automatically overwrite keys with the same name. If a delete operation is performed on the source database during this time, the command will not be synchronized to the destination, resulting in the destination database having more data than the source.

• Why does the precheck verify if the Redis eviction policy is noeviction?

  The default eviction policy (maxmemory-policy) for a Tair (Redis OSS-compatible) instance is volatile-lru. If the destination runs out of memory, eviction is triggered, causing data inconsistency without failing the task. To prevent this, set the destination eviction policy to noeviction. With this policy, writes fail when memory is full, causing the migration task to fail but preventing data loss from eviction. For more information, see Redis data eviction policies.

• Why is there a key named DTS_REDIS_TIMESTAMP_HEARTBEAT in the source database?

  DTS inserts a key with the prefix DTS_REDIS_TIMESTAMP_HEARTBEAT into the source database to record update timestamps. For cluster sources, the key is inserted into each shard. DTS filters out this key during migration, and it expires automatically when the task completes.

• Which commands are supported for incremental data migration?

  • The following commands are supported:

    • APPEND

    • BITOP, BLPOP, BRPOP, BRPOPLPUSH

    • DECR, DECRBY, DEL

    • EVAL, EVALSHA, EXEC, EXPIRE, EXPIREAT

    • FLUSHALL, FLUSHDB

    • GEOADD, GETSET

    • HDEL, HINCRBY, HINCRBYFLOAT, HMSET, HSET, HSETNX

    • INCR, INCRBY, INCRBYFLOAT

    • LINSERT, LPOP, LPUSH, LPUSHX, LREM, LSET, LTRIM

    • MOVE, MSET, MSETNX, MULTI

    • PERSIST, PEXPIRE, PEXPIREAT, PFADD, PFMERGE, PSETEX, PUBLISH

    • RENAME, RENAMENX, RESTORE, RPOP, RPOPLPUSH, RPUSH, RPUSHX

    • SADD, SDIFFSTORE, SELECT, SET, SETBIT, SETEX, SETNX, SETRANGE, SINTERSTORE, SMOVE, SPOP, SREM, SUNIONSTORE

    • ZADD, ZINCRBY, ZINTERSTORE, ZREM, ZREMRANGEBYLEX, ZUNIONSTORE, ZREMRANGEBYRANK, ZREMRANGEBYSCORE

    • XADD, XCLAIM, XDEL, XAUTOCLAIM, XGROUP CREATECONSUMER, XTRIM

  • When using EVAL or EVALSHA to call Lua scripts, DTS cannot confirm whether the script executed successfully because the destination database does not explicitly return an execution result.

Appendix: Advanced settings