Migrate data between self-managed databases on ECS instances - Elastic Compute Service

Data Transmission Service (DTS) is a data streaming service from Alibaba Cloud that supports data transmission between sources, such as Relational Database Management System (RDBMS), NoSQL, and Online Analytical Processing (OLAP) databases. This topic describes how to configure a DTS migration task to migrate data between self-managed databases hosted on Elastic Compute Service (ECS) instances. In this topic, MySQL databases are used.

Background information

DTS supports data migrations between homogeneous and heterogeneous data sources, and provides Extract, Transform, and Load (ETL) features, such as data filtering and database-, table-, and column-level mapping. With DTS, you can perform zero-downtime migrations, allowing the source database to serve requests without interruption, minimizing the impact on your business. For information about the database types supported by DTS, see Data migration.

The following figure shows data migration between self-managed databases hosted on ECS instances.

Prerequisites

The port on which MySQL listens is open in the security groups of the destination ECS instance. The default port is 3306. For more information, see Add a security group rule.

Non-root MySQL database accounts are created on the source and destination ECS instances.

For example, run the following commands to create a MySQL account whose username is dts and password is Test@123456:

# Step 1: Create an account and configure its password.
CREATE USER 'dts'@'%' IDENTIFIED BY 'Test@123456';

# Step 2: Grant privileges.
GRANT ALL PRIVILEGES ON *.* TO 'dts'@'%';

# Step 3: Flush privileges.
FLUSH PRIVILEGES;

Considerations

DTS supports data migration between self-managed databases, but the support may vary for different database versions. Some versions may have limitations. For information about the supported versions and best practices, see Overview of migration solutions.

Procedure

Log on to the DTS console.
In the left-side navigation pane, click Data Migration.
Select the region of the ECS instances and click Create Task.

Configure the source and destination databases.

Specify a task name.
DTS automatically generates a task name. You can also specify a name. To facilitate task identification, specify descriptive name. The specified task name do not need to be unique.

Configure the source database.

DTS supports self-managed databases accessible over the Internet or by using VPN Gateway, Express Connect, or Smart Access Gateway. In this topic, self-managed databases accessible over the Internet are used. For information about how to migrate the data from other types of databases, see Overview of migration solutions.

Parameter	Description
Database Type	The type of the self-managed database hosted on the source ECS instance. Example: MySQL.
Access Method	The access method of the source database. Select an access method based on where the source database is deployed. Example: Self-managed Database on ECS.
Instance Region	The region of the source ECS instance. Example: China (Hangzhou).
Replicate Data Across Alibaba Cloud Accounts	Specifies whether to configure the task to migrate or synchronize data between ECS instances in different Alibaba Cloud accounts. Example: No.
ECS Instance ID	The ID of the source ECS instance.
Port Number	The port on which MySQL listens. The default port is 3306.
Database Account	The non-root account of the MySQL database hosted on the source ECS instance. Example: `dts`. Important The account must be a non-root account. Otherwise, a connectivity test error occurs.
Database Password	The account password.

Configure the destination database.

Parameter	Description
Database Type	The type of the self-managed database hosted on the destination ECS instance. The database type must be the same as that of the self-managed database hosted on the source ECS instance. Example: MySQL.
Access Method	The access method of the destination database. Example: Self-managed Database on ECS.
Instance Region	The region of the destination ECS instance. Example: China (Beijing).
ECS Instance ID	The ID of the destination ECS instance. DTS supports ECS instances deployed in the classic network and virtual private clouds (VPCs).
Port Number	The port on which MySQL listens. The default port is 3306.
Database Account	The non-root account of the MySQL database hosted on the destination ECS instance. Example: `dts`. Note The account must be a non-root account. Otherwise, a connectivity test error occurs.
Database Password	The account password.

In the lower part of the page, click Test Connectivity and Proceed.
If an IP address whitelist is configured for your self-managed database, add the CIDR blocks of DTS servers to the IP address whitelist. Then, click Test Connectivity.
Warning
If the public CIDR blocks of DTS servers are automatically or manually added to the whitelist of a database instance or to the security group rules of an ECS instance, security risks may arise. Therefore, before you use DTS to migrate data, you must understand and acknowledge the potential risks and take preventive measures, including but not limited to the following measures: enhancing the security of your username and password, limiting the ports that are exposed, authenticating API calls, regularly checking the whitelist or security group rules and forbidding unauthorized CIDR blocks, or connecting the database instance to DTS by using Express Connect, VPN Gateway, or Smart Access Gateway.

Configure parameters, such as migration types, migration objects, and the method to migrate triggers.
1. Configure migration types.
  - To perform zero-downtime migration while allowing continuous writes to the source database, select Schema Migration, Full Data Migration, and Incremental Data Migration.
    Note
    If you do not select Incremental Data Migration, avoid writing new data to the source database during the migration to ensure data consistency.
  - To migrate data during downtime or when no new data is written to the source database, select Schema Migration and Full Data Migration.
2. Configure migration objects.
  In the Source Objects list, select the database objects you want to migrate, such as databases, tables, or columns. Then, click the icon to add them to the Selected Objects list.
  Note
  By default, object names remain unchanged after migration from the source MySQL database to the destination MySQL database. If an object has a different name in the destination database, use the object name mapping feature in DTS. For more information, see Object name mapping.
Complete the other configurations as prompted. Perform a precheck, purchase a DTS instance, complete the payment, and then start the migration task.
Before the migration task starts, DTS checks items, such as connectivity, permissions, and log formats. The following figure shows that the precheck is successful.
After the precheck is complete, you can view the migration task's status and progress in the Data Migration Tasks section on the Data Migration page.