This topic describes how to migrate data from an AWS ElastiCache for Redis instance to an Alibaba Cloud ApsaraDB for Redis instance.

Prerequisites

The ApsaraDB for Redis instance runs the ApsaraDB for Redis Community Edition in Redis 4.0 or 5.0 or the ApsaraDB for Redis Enhanced Edition (Tair) in Redis 5.0. If no ApsaraDB for Redis instances are created, you must create an ApsaraDB for Redis instance. For more information, see Step 1: Create an ApsaraDB for Redis instance.

Introduction to redis-shake

Redis-shake is an open source tool that is developed by Alibaba Cloud. Redis-shake is easy to deploy and can be used to efficiently parse (decode mode), restore (restore mode), back up (dump mode), and synchronize (sync or rump mode) the data of Redis databases. In this topic, the restore mode of redis-shake is used to restore the data of the self-managed Redis database from an RDB file to the ApsaraDB for Redis instance.

Note

Precautions

  • Before you start the migration, we recommend that you stop the write operations on the AWS ElastiCache for Redis cluster.
  • Before you start the migration, we recommend that you back up the AWS ElastiCache for Redis cluster and prepare a migration schedule that minimizes downtime.
  • If the data eviction policy (maxmemory-policy) of the destination database is not set to noeviction, data may become inconsistent between the source and destination databases. For more information about the data eviction policy, see How does ApsaraDB for Redis evict data by default?
  • After you run the info command to query the keys in the destination database, you may find that the destination database contains fewer keys than the source database. This is caused by the key expiration mechanism of Redis. This situation occurs if the source database contains keys that are not deleted upon expiration.
    Note The numbers of keys that do not have a validity period in the source and destination databases are the same.

Step 1: Export the data of the AWS ElastiCache for Redis cluster as an RDB file

  1. Create a backup and store the generated backup file to a different AWS ElastiCache for Redis cluster. Log on to the AWS ElastiCache console. In the left-side navigation pane, click Redis. On the page that appears, select the AWS ElastiCache for Redis cluster and click Backup. In the dialog box that appears, set the Resource Name parameter to the name of the AWS ElastiCache for Redis cluster, enter the name of the backup in the Backup Name field, and then click Create Backup.
  2. Export the backup file to an AWS S3 bucket as an RDB object. In the left-side navigation pane, click Backups. On the page that appears, select the backup file and click Copy above the file list. In the dialog box that appears, configure the New backup name and Target S3 location parameters and click Copy.
  3. View the RDB object in the AWS S3 bucket that you specify.
  4. Download the RDB object from the AWS S3 bucket to your computer as an RDB file.

Step 2: Migrate the RDB file to the ApsaraDB for Redis instance by using redis-shake

  1. Perform the following operations based on the installation location of redis-shake:
    Note We recommend that you install redis-shake on an Elastic Compute Service (ECS) instance. You can connect the ECS instance and the ApsaraDB for Redis instance over a virtual private cloud (VPC) to increase security and reduce network latencies.
    Installation location of redis-shake Operation
    (Recommended) ECS instance
    1. Make sure that the ECS instance and the ApsaraDB for Redis instance belong to the same VPC. They share the same VPC ID in the Basic Information section.
      Note
    2. Obtain the internal IP address of the ECS instance. For more information, see How do I query IP addresses of ECS instances.
    3. Add the internal IP address of the ECS instance to the whitelist of the ApsaraDB for Redis instance. For more information, see Configure whitelists.
    On-premises machine
    1. By default, an ApsaraDB for Redis instance provides only an internal endpoint. You must apply for a public endpoint when you want to connect to an ApsaraDB for Redis instance over the Internet. For more information, see Apply for a public endpoint.
    2. Run the curl ipinfo.io |grep ip command on the on-premises machine to obtain the public IP address. The returned result is shown in the following figure.View public IP address results
    3. Add the public IP address of the on-premises machine to the whitelist of the ApsaraDB for Redis instance. For more information, see Configure whitelists.
  2. Install redis-shake.
    1. Log on to the host where you want to install redis-shake. The host may be an ECS instance or an on-premises machine.
    2. Run the following command to download the redis-shake file.
      wget 'http://docs-aliyun.cn-hangzhou.oss.aliyun-inc.com/assets/attach/120287/cn_zh/1608173646665/redis-shake-v2.0.3.tar.gz'
      Note This example shows how to install redis-shake 2.0.3. You can also install redis-shake of other versions. For more information, see RedisShake.
    3. Run the following command to decompress the redis-shake file:
      tar xzf redis-shake-v2.0.3.tar.gz
  3. Start the migration on the ECS instance or the on-premises machine.
    1. Download the RDB file to the ECS instance or the on-premises machine.
      Note You can run the wget <The URL from which you can download the RDB file> command to download the RDB file to the ECS instance. You can also download the RDB file to the on-premises machine and then use MobaXterm Personal Edition to upload the RDB file from the on-premises machine to the ECS instance over SFTP.
    2. Run the following command to open and edit the redis-shake.conf file in the directory that is generated after you decompress the redis-shake software package:
      cd redis-shake-v2.0.3/ && vim redis-shake.conf
      Note After you run the command, the system opens an editor. You can type a to enter the edit mode.
      Table 1. Parameter description
      Parameter Required Description Example
      source.rdb.input Yes The save path of the RDB file. You can specify a relative path or an absolute path.
      Note If you need to specify more than one RDB file, separate the save paths of the RDB files with semicolons (;).
      /root/tools/RedisShake/demo.rdb
      target.address Yes The endpoint and port number of the ApsaraDB for Redis instance. Separate the endpoint and port number with a colon (:). For more information about how to obtain the endpoint and port number of an ApsaraDB for Redis instance, see View endpoints.
      • If you install redis-shake on an ECS instance and connect the ECS instance and the ApsaraDB for Redis instance over a VPC, obtain the internal endpoint and port number of the ApsaraDB for Redis instance.
      • If you install redis-shake on an on-premises machine and connect the on-premises machine and the ApsaraDB for Redis instance over the Internet, obtain the public endpoint and port number of the ApsaraDB for Redis instance.
      r-bp1wcw2rlw76acc5k****.redis.rds.aliyuncs.com:6379
      target.password_raw Yes The username and password of the account that has the Read/Write permissions on the ApsaraDB for Redis instance. Separate the username and password with a colon (:). For more information about how to create an account, see Create and manage database accounts. testaccount:Rp829dlwa
      key_exists No The data write policy that is applied if the keys in AWS ElastiCache for Redis cluster are the same as the keys in the ApsaraDB for Redis instance. Valid values:
      • rewrite: The keys in the AWS ElastiCache for Redis cluster overwrite the keys with identical names in the ApsaraDB for Redis instance.
      • none: Redis-shake stops running, and a message that indicates conflicting keys appears. This is the default value.
      • ignore: Redis-shake skips the conflicting keys, retains the data of the ApsaraDB for Redis instance, and continues to migrate the other data.
      rewrite
      parallel No The number of concurrent threads that are invoked by redis-shake to perform the migration. You can increase the value of this parameter to accelerate the migration.
      Note The default value is 32. The minimum value is 1. The maximum value varies based on the performance of the server on which redis-shake is installed.
      32
      Note You do not need to configure other parameters unless otherwise specified. For more information, see the comments on each parameter in the redis-shake.conf file.
    3. Press Esc to exit the edit mode. Then, type :wq and press Enter to save the .redis-shake.conf file and exit the editor.
    4. Run the following command to start redis-shake and migrate data:
      ./redis-shake.linux -type=restore -conf=redis-shake.conf
      Redis-shake displays the operational log on the screen.
      Note For more information about the causes of and solutions to possible errors, see FAQ.
  4. When restore: rdb done appears, press Ctrl+C to stop redis-shake.
    Figure 1. Example