All Products
Search
Document Center

Elastic Compute Service:Migrate data between self-managed databases on ECS instances

Last Updated:Mar 25, 2024

Data Transmission Service (DTS) is a data streaming service provided by Alibaba Cloud that supports data transmission between data sources, such as Relational Database Management System (RDBMS), NoSQL, and Online Analytical Processing (OLAP) databases. This topic describes how to configure a DTS migration task to migrate data between self-managed databases that are hosted on Elastic Compute Service (ECS) instances. In this topic, MySQL databases are used.

Prerequisites

  • The port on which MySQL listens is opened in the security groups of the destination ECS instance. The default port on which MySQL listens is 3306. For more information, see Add a security group rule.

  • Non-root accounts are created for the MySQL databases on the source and destination ECS instances.

    For example, you can run the following command to create a MySQL account whose username is dts and password is 123456:

    grant all on *.*  to 'dts'@'%'  IDENTIFIED BY '123456';

Background information

DTS supports data migration between homogeneous and heterogeneous data sources. It also supports Extract, Transform, Load (ETL) features such as data mapping at three levels (databases, tables, and columns) and data filtering. You can use DTS for zero-downtime migration to minimize impacts on your business. During data migration, the source database continues to provide services as expected. For information about the database types supported by DTS, see Data migration.

Procedure

  1. Log on to the DTS console.

  2. In the left-side navigation pane, click Data Migration.

  3. Select the region where the ECS instances reside and click Create Task.

  4. Configure the source and destination databases.

    1. Configure a task name.

      DTS automatically generates a task name. We recommend that you specify a descriptive name to facilitate task identification. You do not need to specify a unique task name. You can use the default name or specify a name.

    2. Configure the source database.

      Note

      If you created Data Management (DMS) database instances, you can select an existing instance to configure the source database.

      DTS supports self-managed databases that can be accessed over the Internet or by using VPN Gateway, Express Connect, or Smart Access Gateway. In this topic, self-managed databases that are accessed over the Internet are used. For information about how to migrate the data of other types of databases, see Overview of data migration scenarios.

      Parameter

      Description

      Database Type

      The type of the self-managed database hosted on the source ECS instance. Example: MySQL.

      Access Method

      The access method of the source database. Select an access method based on the deployment location of the source database. Example: Self-managed Database on ECS.

      Instance Region

      The region where the self-managed database on the source ECS instance resides. Example: China (Hangzhou).

      Replicate Data Across Alibaba Cloud Accounts

      Specifies whether to configure the task to migrate or synchronize data between instances that belong to different Alibaba Cloud accounts. Example: No.

      ECS Instance ID

      The ID of the source ECS instance.

      Port Number

      The port on which MySQL listens. The default port is 3306.

      Database Account

      The non-root account that is used to access the MySQL database on the source ECS instance.

      Important

      The account must be a non-root account. Otherwise, an error occurs when you test connectivity.

      Database Password

      The password of the non-root account.

    3. Configure the destination database.

      Note

      If you created DMS database instances, you can select an existing instance to configure the destination database.

      Parameter

      Description

      Database Type

      The type of the self-managed database hosted on the destination ECS instance. This database type must be the same as the type of the self-managed database hosted on the source ECS instance. Example: MySQL.

      Access Method

      The method used to access the destination database. Example: Self-managed Database on ECS.

      Instance Region

      The region where the destination ECS instance resides. Example: China (Beijing).

      ECS Instance ID

      The ID of the destination instance. DTS supports ECS instances that reside in the classic network and in virtual private clouds (VPCs).

      Port Number

      The port on which MySQL listens. The default port is 3306.

      Database Account

      The non-root account that is used to access the MySQL database on the destination ECS instance.

      Note

      The account must be a non-root account. Otherwise, an error occurs when you test connectivity.

      Database Password

      The password of the non-root account.

    4. In the lower part of the page, click Test Connectivity and Proceed.

    5. If an IP address whitelist is configured for your self-managed database, add the CIDR blocks of DTS servers to the IP address whitelist. Then, click Test Connectivity and Proceed.

      Warning

      If the public CIDR blocks of DTS servers are automatically or manually added to the IP address whitelist of a database instance or to the security group rules of an ECS instance, security risks may arise. Therefore, before you use DTS to migrate data, you must understand and acknowledge the potential risks and take preventive measures, including but not limited to the following measures: enhancing the security of your account and password, limiting the ports that are exposed, authenticating API calls, regularly checking the IP address whitelist or ECS security group rules and forbidding unauthorized CIDR blocks, and connecting the database to DTS by using Express Connect, VPN Gateway, or Smart Access Gateway.

  5. Configure migration types and migration objects.

    1. Configure migration types.

      • To perform zero-downtime data migration, select Schema Migration, Full Data Migration, and Incremental Data Migration.

        Note

        If you do not select Incremental Data Migration, do not write new data to the source database during data migration to ensure data consistency.

      • To perform full data migration, select Schema Migration and Full Data Migration.

    2. Configure migration objects.

      In the Available list, select the database objects that you want to migrate, such as databases, tables, or columns. Then, click the 2023-09-08_17-04-13.png icon to add the objects to the Selected Objects list.

      Note

      By default, the object names remain the same as those in the source MySQL database after the objects are migrated to the destination MySQL database. If a migration object has different names in the source and destination databases, you must use the object name mapping feature provided by DTS. For more information, see Object name mapping.

  6. Click Precheck.

    Before the migration task starts, DTS checks items such as connectivity, permissions, and log formats. The following figure shows that the precheck is successful.

    After the precheck is complete, you can view the status and progress of the migration task in the Migration Tasks section.