This topic describes how to migrate the data of a self-managed Redis database from a backup file in the RDB format to an ApsaraDB for Redis instance by using redis-shake.

Prerequisites

  • The self-managed Redis database runs Redis 2.8, 3.0, 3.2, 4.0, or 5.0.
    Note If the self-managed Redis database and the ApsaraDB for Redis instance run different Redis versions, we recommend that you create a pay-as-you-go ApsaraDB for Redis instance to test the compatibility between the two versions. If no compatibility issues are found, you can change the billing method of the ApsaraDB for Redis instance to subscription and migrate the data of the self-managed Redis database to the ApsaraDB for Redis instance. For more information about how to change the billing method of an ApsaraDB for Redis instance, see Change the billing method to subscription.
  • The ApsaraDB for Redis instance runs the ApsaraDB for Redis Community Edition in Redis 4.0 or 5.0 or the ApsaraDB for Redis Enhanced Edition (Tair) in Redis 5.0. For more information about how to create an ApsaraDB for Redis instance, see Step 1: Create an ApsaraDB for Redis instance.

Introduction to redis-shake

Redis-shake is an open source tool that is developed by Alibaba Cloud. Redis-shake is easy to deploy and can be used to efficiently parse (decode mode), restore (restore mode), back up (dump mode), and synchronize (sync or rump mode) the data of Redis databases. In this topic, the restore mode of redis-shake is used to restore the data of the self-managed Redis database from an RDB file to the ApsaraDB for Redis instance.

Note

Precautions

  • If the data eviction policy (maxmemory-policy) of the destination database is not set to noeviction, data may become inconsistent between the source and destination databases. For more information about the data eviction policy, see How does ApsaraDB for Redis evict data by default?
  • After you run the info command to query the keys in the destination database, you may find that the destination database contains fewer keys than the source database. This is caused by the key expiration mechanism of Redis. This situation occurs if the source database contains keys that are not deleted upon expiration.
    Note The numbers of keys that do not have a validity period in the source and destination databases are the same.

Procedure

  1. Perform the following operations based on the installation location of redis-shake:
    Note We recommend that you install redis-shake on an Elastic Compute Service (ECS) instance. You can connect the ECS instance and the ApsaraDB for Redis instance over a virtual private cloud (VPC) to increase security and reduce network latencies.
    Installation location of redis-shake Operation
    (Recommended) ECS instance
    1. Make sure that the ECS instance and the ApsaraDB for Redis instance belong to the same VPC. They share the same VPC ID in the Basic Information section.
      Note
    2. Obtain the internal IP address of the ECS instance. For more information, see How do I query IP addresses of ECS instances.
    3. Add the internal IP address of the ECS instance to the whitelist of the ApsaraDB for Redis instance. For more information, see Configure whitelists.
    On-premises machine
    1. By default, an ApsaraDB for Redis instance provides only an internal endpoint. You must apply for a public endpoint when you want to connect to an ApsaraDB for Redis instance over the Internet. For more information, see Apply for a public endpoint.
    2. Run the curl ipinfo.io |grep ip command on the on-premises machine to obtain the public IP address. The returned result is shown in the following figure.View public IP address results
    3. Add the public IP address of the on-premises machine to the whitelist of the ApsaraDB for Redis instance. For more information, see Configure whitelists.
  2. Install redis-shake.
    1. Log on to the host where you want to install redis-shake. The host may be an ECS instance or an on-premises machine.
    2. Run the following command to download the redis-shake file.
      wget 'http://docs-aliyun.cn-hangzhou.oss.aliyun-inc.com/assets/attach/120287/cn_zh/1608173646665/redis-shake-v2.0.3.tar.gz'
      Note This example shows how to install redis-shake 2.0.3. You can also install redis-shake of other versions. For more information, see RedisShake.
    3. Run the following command to decompress the redis-shake file:
      tar xzf redis-shake-v2.0.3.tar.gz
  3. Start the migration on the ECS instance or the on-premises machine.
    1. Copy the RDB file that contains the backup data of the self-managed Redis database to the ECS instance or the on-premises machine. In this example, the RDB file is stored in the following path: /root/redisdump.
    2. Run the following command to open and edit the redis-shake.conf file in the directory that is generated after you decompress the redis-shake software package:
      cd redis-shake-v2.0.3/ && vim redis-shake.conf
      Note After you run the command, the system opens an editor. You can type a to enter the edit mode.
      Table 1. Parameter description
      Parameter Required Description Example
      source.rdb.input Yes The save path of the RDB file. You can specify a relative path or an absolute path.
      Note If you need to specify more than one RDB file, separate the save paths of the RDB files with semicolons (;).
      /root/redisdump/dump.rdb
      target.address Yes The endpoint and port number of the ApsaraDB for Redis instance. Separate the endpoint and port number with a colon (:). For more information about how to obtain the endpoint and port number of an ApsaraDB for Redis instance, see View endpoints.
      • If you install redis-shake on an ECS instance and connect the ECS instance and the ApsaraDB for Redis instance over a VPC, obtain the internal endpoint and port number of the ApsaraDB for Redis instance.
      • If you install redis-shake on an on-premises machine and connect the on-premises machine and the ApsaraDB for Redis instance over the Internet, obtain the public endpoint and port number of the ApsaraDB for Redis instance.
      r-bp1wcw2rlw76acc5k****.redis.rds.aliyuncs.com:6379
      target.password_raw Yes The username and password of the account that has the Read/Write permissions on the ApsaraDB for Redis instance. Separate the username and password with a colon (:). For more information about how to create an account, see Create and manage database accounts. testaccount:Rp829dlwa
      key_exists No The data write policy that is used when the keys in the self-managed Redis database are the same as the keys in the ApsaraDB for Redis instance. Valid values:
      • rewrite: The keys in the self-managed Redis database overwrite the keys with identical names in the ApsaraDB for Redis instance.
      • none: Redis-shake stops running, and a message that indicates conflicting keys appears. This value is the default value.
      • ignore: Redis-shake skips the keys in the self-managed Redis database, retains the keys with identical names in the ApsaraDB for Redis instance, and continues to migrate the other data.
      rewrite
      parallel No The number of concurrent threads that are invoked by redis-shake to perform the migration. You can increase the value of this parameter to accelerate the migration.
      Note The default value is 32. The minimum value is 1. The maximum value varies based on the performance of the server on which redis-shake is installed.
      32
      Note You do not need to configure other parameters unless otherwise specified. For more information, see the comments on each parameter in the redis-shake.conf file.
    3. Press Esc to exit the edit mode. Then, type :wq and press Enter to save the redis-shake.conf file and exit the editor.
    4. Run the following command to start redis-shake and migrate data:
      ./redis-shake.linux -type=restore -conf=redis-shake.conf
      Redis-shake displays the operational log on the screen.
      Note For more information about the causes of and solutions to possible errors, see FAQ.
  4. Wait until the migration is complete. Then, end the migration task.
    1. Check the log data and wait until the migration task completes incremental data synchronization.
      2021/06/18 10:44:25 [INFO] restore from '[/root/redisdump/dump.rdb]' to '[r-bp1bjmqwlelmkhy****.redis.rds.aliyuncs.com:6379]'
      2021/06/18 10:44:25 [INFO] routine[0] starts restoring data from /root/redisdump/dump.rdb to [r-bp1bjmqwlelmkhy5ry933.redis.rds.aliyuncs.com:6379]
      2021/06/18 10:44:26 [INFO] routine[0] total = 11.15MB -       6.35MB [ 56%]  entry=5692
      2021/06/18 10:44:27 [INFO] routine[0] total = 11.15MB -      11.15MB [100%]  entry=10001
      2021/06/18 10:44:27 [INFO] routine[0] restore: rdb done
      2021/06/18 10:44:27 [INFO] restore from '[/root/redisdump/dump.rdb]' to '[r-bp1bjmqwlelmkhy****.redis.rds.aliyuncs.com:6379]' done
      2021/06/18 10:44:27 [INFO] Enabled http stats, set status (incr), and wait forever.
      Note When restore: rdb done appears in the operational log, the migration is complete.
    2. Press Ctrl+C to stop redis-shake.
  5. Optional:Verify the data in the ApsaraDB for Redis instance. For more information, see Verify data after migration.