You can use the sync mode of redis-shake to migrate data from an ApsaraDB for Redis
instance to a user-created Redis database in real time. This enables you to migrate
the data of a user-created Redis database to the cloud.
Prerequisites
The version of the user-created Redis database is Redis 2.8, 3.0, 3.2, 4.0, or 5.0.
Note If you want to migrate data between different versions of Redis databases, for example,
from Redis 2.8 to Redis 4.0, we recommend that you create a pay-as-you-go ApsaraDB
for Redis instance to verify the compatibility. If the compatibility verification
is passed, you can convert the billing method of the instance to subscription. For
more information about how to convert the billing method, see Public bandwidth.
The ApsaraDB for Redis instance must meet the following requirements:
The instance is of Community Edition (Redis 4.0 or 5.0) or Enhanced Edition (Redis
5.0).
Redis-shake is an open source Linux-based tool developed by Alibaba Cloud. You can use this flexible
and efficient tool to parse (decode mode), restore (restore mode), back up (dump mode),
and synchronize (sync or rump mode) data on ApsaraDB for Redis instances. The sync
mode of redis-shake supports full data migration and incremental data migration. The
detailed migration process is shown in the following figure:
Figure 1. Redis-shake data migration diagram
Note The data synchronization feature of Data Transmission Service (DTS) supports more
data synchronization use cases. For more information, see Overview.
Procedure
Perform the following operations based on redis-shake installation location:
Note We recommend that you install redis-shake on the host where the user-created database
resides. This reduces network latency and prevents connection failures caused by the
firewall settings of the user-created database.
Make sure that the ECS instance and the ApsaraDB for Redis instance are deployed in
the same VPC. The ECS instance and ApsaraDB for Redis instance must have the same
VPC ID.
Note If they are deployed in different VPCs, you must change the VPC to which the ECS instance
belongs. For more information, see Change the VPC of an ECS instance.
Add the internal IP address of the ECS instance to the whitelist of the ApsaraDB for
Redis instance. For more information, see Set IP address whitelists.
On-premises host
By default, an ApsaraDB for Redis instance provides only an internal endpoint. You
must apply for a public IP address when you want to connect to an ApsaraDB for Redis
instance over the Internet. For more information, see Apply for a public endpoint for an ApsaraDB for Redis instance.
Run the curl ipinfo.io |grep ip command on the on-premises host to obtain the public IP address. The returned result
is shown in the following figure:
Add the public IP address of the on-premises host to the whitelist of the ApsaraDB
for Redis instance. For more information, see Set IP address whitelists.
Install redis-shake.
Log on to the host where redis-shake is to be installed, such as an ECS instance or an on-premises host.
Run the following command to download the redis-shake file.
Note This topic uses version 2.0.3 as an example to demonstrate the operation process.
You can also install other redis-shake versions. For more information, see RedisShake.
Run the following command to decompress the redis-shake file:
tar xzf redis-shake-v2.0.3.tar.gz
Migrate data on the host where redis-shake is installed.
Run the following command to access the decompressed directory and modify the configuration
file:
cd redis-shake-v2.0.3/ && vim redis-shake.conf
Note After the command is executed, the system will open an editing interface. Enter a to enter the editing mode.
Table 1. Parameters
Parameter
Description
Example
parallel (optional)
The number of concurrent threads for redis-shake to perform the migration. You can
modify this value to improve synchronization performance.
Note The default value is 32. The minimum value is 1. The maximum value depends on the
performance of the server where redis-shake resides.
32
source.type (required)
Select based on the architecture of the source database (user-created database). Valid
values:
standalone: master/replica architecture.
cluster: cluster architecture.
standalone
source.address (required)
The connection address and port number of the source database. Separate the connection
address and port number with a colon (:).
Note
If the source database and redis-shake are deployed on the same host, enter 127.0.0.1:6379.
If the source database is a cluster instance, you must add the prefix master@ to the connection address, for example, master@127.0.0.1:6379.
127.0.0.1:6379
source.password_raw (optional)
The password of the source database. If no password is set for the destination database,
you do not need to specify a password.
Rp829dlwa
target.type (required)
Select based on the architecture of the destination ApsaraDB for Redis instance. Valid
values:
The connection address and port number of the destination ApsaraDB for Redis instance.
Separate the connection address and port number with a colon (:). For more information
about how to obtain the connection address and port number of the ApsaraDB for Redis
instance, see View endpoints.
The ECS instance is connected through a virtual private cloud (VPC): You must obtain
the private endpoint of the ApsaraDB for Redis instance.
The on-premises host is connected to the ApsaraDB for Redis instance over the Internet:
you must obtain the public endpoint of the ApsaraDB for Redis instance.
Note If the destination instance is of cluster architecture, you must connect to the instance
deployed in a VPC by using a private endpoint. You must add the prefix master@ to the connection endpoint, for example, master@r-bp1mfnrflszg75w****.redis.rds.aliyuncs.com:6379. For more information about how to request a private endpoint, see View or apply for an endpoint.
The account with read and write permissions and password of the destination ApsaraDB for Redis instance. Separate
the account and password with a colon (:). For more information about how to create
an account, see Create accounts and databases for an ApsaraDB RDS for MySQL instance.
testaccount:Rp829dlwa
target.db (optional)
Migrates data from all databases in the source database to the specified databases
of the destination database. Valid values: 0 to 15.
Note The default value -1 indicates that the feature is disabled.
-1
key_exists (optional)
The data writing policy that is adopted when the keys in the source database are the
same as those in the destination database. Valid values:
rewrite: specifies whether to overwrite the existing keys in the destination database that
are the same as those in the source database.
none: This is the default value. Redis-shake stops running and a message appears that
indicates conflicting keys.
ignore: skips the current key, retains the data in the destination database, and continues
the data migration.
rewrite
filter.db.whitelist (optional)
The names of the databases to be migrated. Separate multiple database names with semicolons
(;). The default value null indicates that all databases are to be migrated.
0;1
filter.db.blacklist (optional)
The names of databases that do not need to be migrated. This is a blacklist. Separate
multiple database names with semicolons (;). By default, this parameter is not specified
and no blacklist is configured.
1;2
Note You do not need to configure other parameters unless otherwise required. For more
information, see the comments on each parameter in the redis-shake.conf file.
Press the ESC key to exit the editing mode and enter :wq to save and close the file.
Run the following command to start redis-shake and migrate data:
forwardCommands: the number of commands that are sent from the source database.
filterCommands: the number of commands that are filtered. For example, you set to filter specific
databases in the configuration file of redis-shake.
writeBytes: the number of bytes that are sent from the source database.
Pause writing data to the source database and wait until the value of the writeBytes parameter in the returned log remains 0 for multiple times in succession, and use
Ctrl+C to stop running redis-shake.
Figure 2. Stop running example
Note In this case, the data in the source databases and destination databases is consistent.
You can switch the database service from the on-premises Redis database to the ApsaraDB
for Redis instance.