Use redis-shake to migrate data

Prerequisites

• An ApsaraDB for Redis instance is created as the destination of data migration.
• An Elastic Compute Service (ECS) instance installed with the Linux operating system is created for running redis-shake.
• The IP address of the ECS instance is added to the whitelist of the destination ApsaraDB for Redis instance.

Background

Redis-shake is an open-source tool developed by Alibaba Cloud. You can use it to parse (decode mode), recover (restore mode), back up (dump mode), and synchronize (sync/rump mode) Redis data. In sync mode, redis-shake runs a SYNC or PSYNC command to synchronize full or incremental data from the source Redis to the destination Redis. Incremental synchronization automatically starts after full synchronization is completed. The sync mode applies to scenarios such as data migration from on-premises Redis to the cloud, data synchronization between on-premises Redis and ApsaraDB for Redis, and data synchronization between on-premises Redis databases. This topic describes how to use the sync mode of redis-shake to migrate data from on-premises Redis to ApsaraDB for Redis.

Procedure

1. Log on to the ECS instance that can access the destination ApsaraDB for Redis instance.
2. Download the redis-shake tool in the ECS instance.
   Note We recommend that you download the latest version.
3. Run the following command to decompress the downloaded redis-shake.tar.gz package:

   # tar -xvf redis-shake.tar.gz

   Note In the decompressed folder, the redis-shake file is a binary file that can be run in the 64-bit Linux operating system. The redis-shake.conf file is the configuration file of redis-shake. You need to modify this configuration file in the next step.
4. Modify the redis-shake.conf file. The following table describes the parameters for the sync mode of redis-shake.

   Table 1. Parameters for the sync mode of redis-shake

   Parameter	Description	Example
   source.address	The connection address and service port of the source on-premises Redis instance.	xxx.xxx. 1.10:6379
   source.password_raw	The password of the source on-premises Redis instance.	SourcePass233 Note If you synchronize data from ApsaraDB for Redis, you need to use a database account that has the Replicate permission to connect to the source ApsaraDB for Redis instance. In this case, set this parameter in the following format: account:password.
   target.address	The connection address and service port of the destination ApsaraDB for Redis instance.	r-bp1xxxxxxxxxxxxx.redis.rds.aliyuncs.com:6379
   target.password_raw	The password of the destination ApsaraDB for Redis instance.	TargetPass233
   rewrite	Specifies whether to overwrite the existing keys in the destination ApsaraDB for Redis instance that are identical to those in the RDB file. Valid values: true false Note Default value: true. We recommend that you back up the valid data of the destination ApsaraDB for Redis instance before migration. If you set this parameter to false and any keys are duplicate in the source and destination databases, an error message is returned.	true
   target.db	The database to which the data is migrated in the destination ApsaraDB for Redis instance. For example, to migrate data from the source on-premises Redis instance to DB10 of the destination ApsaraDB for Redis instance, set this parameter to 10. If you set this parameter to a value less than 0, data is migrated to DB0.	0
   parallel	The number of concurrent threads used to synchronize the RDB file. More concurrent threads improve synchronization performance. Note Minimum value: 1. Maximum value: depends on the server performance. Recommended value: 64.	64

5. Run the following command to migrate data:

   # ./redis-shake -type=sync -conf=redis-shake.conf

   Note You must run this command in the same directory as the redis-shake and redis-shake.conf files. Otherwise, you need to specify the correct file path in the command.
6. View synchronization logs to check the synchronization status. When sync rdb done appears, full synchronization is completed and incremental synchronization starts.

   

   Note

   • In log entries after sync rdb done, +forward=0 indicates that no data is written to the source database and no incremental data is being synchronized. You can monitor the status of data synchronization to determine the best time to switch databases.
   • After migration is completed, you can use the redis-full-check tool to check whether the data is consistent between the source and destination databases. For more information, see Verify data after migration.