Data Transmission Service:Configure a data integration task between RDS for MySQL instances

Prerequisites

Ensure you have created the source and destination ApsaraDB RDS for MySQL instances. For more information, see (Deprecated, redirected to "Step 1") Create an ApsaraDB RDS for MySQL instance.

Background information

The data integration feature of Data Transmission Service (DTS) lets you periodically migrate the schema and existing data from a source database to a destination database based on a scheduling policy. This helps you build a more flexible data warehouse. For more information, see Scheduling policies for data integration.

Notes

Duplicate data may be generated in the destination database if the source database has no primary key, unique constraint, or columns with unique values.

Limitations

• DTS supports schema migration for tables, views, triggers, stored procedures, and stored functions.
  Note During schema migration, DTS changes the DEFINER for views, stored procedures, or functions to INVOKER.
• DTS does not migrate user information. To call views, stored procedures, or functions in the destination, grant read and write permissions to the caller.

Billing

Procedure

This topic describes the configuration procedure, using ApsaraDB RDS for MySQL instances in the China (Qingdao) region as an example.

1. Log on to the DTS console.
   Note

   • If you are automatically redirected from the DTS console to the Data Management (DMS) console, click the icon in the lower-right corner and then click to return to the previous version of the DTS console.
   • If you are on the new DTS console, click the icon in the lower-right corner to return to the previous version of the DTS console.
2. In the left-side navigation pane, click Data Integration.
3. At the top of the Data Integration Tasks page, select the region where the destination instance is located. This example uses China (Qingdao).
4. In the upper-right corner of the page, click Create data integration task.
5. Configure connection settings for the source and destination databases.

   Section	Parameter	Description
   N/A	Task name	DTS automatically generates a task name. Specify a descriptive name for easy identification. The name does not need to be unique.
   Source Database	Instance type	Select RDS Instance.
   	Region	Select the region of the source RDS instance. This example uses China (Qingdao).
   	Database account	Enter the database account for the source RDS instance. This account must have read and write permissions. Note For more information about how to create a database account and grant permissions, see Create an ApsaraDB RDS for MySQL account.
   	Database password	Enter the password for the database account. Note After you configure the source database information, you can click Database Password next to Test Connectivity to verify the settings. If the settings are valid, the Passed message is displayed. If the test fails, a Failed message appears. Click Failed next to the Diagnose message and follow the prompts to modify the source database information.
   	Connection method	Select Non-encrypted or SSL-encrypted. If you select SSL-encrypted, you must enable SSL encryption for the RDS instance in advance. For more information, see Configure SSL encryption. Note The Encryption parameter is available only in the China (Hong Kong) region and Chinese mainland regions.
   Destination Database	Instance type	Select RDS Instance.
   	Region	Select the region of the destination RDS instance. This example uses China (Qingdao).
   	Database account	Enter the database account for the destination RDS instance. This account must have read and write permissions. Note For more information about how to create a database account and grant permissions, see Create an ApsaraDB RDS for MySQL account.
   	Database password	Enter the password for the database account. Note After you configure the destination database information, you can click Database Password next to Test Connectivity to verify the settings. If the settings are valid, the Passed message is displayed. If the test fails, a Failed message appears. Click Failed next to the Diagnose message and follow the prompts to modify the destination database information.
   	Connection method	Select Non-encrypted or SSL-encrypted. If you select SSL-encrypted, you must enable SSL encryption for the RDS instance in advance. For more information, see Configure SSL encryption. Note The Encryption parameter is available only in the China (Hong Kong) region and Chinese mainland regions.

6. After you complete the configuration, click Add whitelist and go to next step in the lower-right corner.
7. Configure a periodic scheduling policy and select the migration objects.
   1. Configure the periodic scheduling policy.

      Parameter	Description
      Migration Types	Select Schema Migration and Full data migration. During the first migration, DTS migrates the schema and existing data for the selected migration objects.
      Scheduling policy	Full copy mode Resynchronization mode Timestamp-based incremental mode Note For more information about the description, limits, and scenarios of scheduling policies, see Scheduling policies.
      Effective dates	Click the icon and select the start and end dates for the schedule. The maximum duration is 365 days.
      Scheduling Cycle	Select Days or Weeks and set a specific point in time for the schedule. Note During full data migration, DTS consumes a specific amount of read and write resources from the source and destination databases, which may increase the database load. Evaluate the performance of the source and destination databases and schedule the task during off-peak hours. After the configuration is complete, you can click View scheduled times next to Task failure policy to verify the schedule.
      Task failure policy	Select one of the following options based on your business requirements: Stop subsequent scheduled tasks upon a single task failure Ignore failure and proceed with next scheduled task

   2. Select the migration objects.

      Parameter

      Description

      Migration objects

      In the Available box, click the objects that you want to migrate, such as databases, tables, or columns, and then click the icon to move the objects to the Selected Objects box. Note If you set Scheduling policy to Timestamp-based incremental mode, you cannot select an entire database as the migration object. You must select specific tables or columns. Otherwise, you cannot specify the timestamp column in the next step. By default, the names of migrated objects remain unchanged after migration. If you want to change the names of the migrated objects in the destination instance, use the object name mapping feature. For more information, see Object name mapping. If you use the object name mapping feature, other objects that depend on the renamed object may fail to be migrated.

   3. In the lower-right corner of the page, click Next.
      Note This step is required only if you set Scheduling policy to Timestamp-based incremental mode.

      In the Advanced settings step, specify a timestamp column for each table that you want to migrate. For example, for the order table in the dtstestdata database, select ordertime from the Timestamp column drop-down list.

8. After you complete the configuration, click Precheck and Start in the lower-right corner of the page.

   Note

   • Before the migration task starts, DTS runs a precheck. The task can start only after it passes the precheck.

   • If the precheck fails, click the icon next to the failed item to view details.

     • Fix the issues as prompted and run the precheck again.

     • If you do not need to fix the warning items, you can select Ignore and then click Ignore Warnings and Rerun Precheck to run the precheck again.

9. After the task passes the precheck, click Next.

10. In the Confirm Settings dialog box that appears, select a Instance Class and select the Data Transmission Service (pay-as-you-go) Service Terms checkbox.

11. Click Purchase and start. The data integration task starts at the next scheduled time.
    After the task starts, the task details page shows that the task status is Running and the precheck status is Passed. The page also displays the progress of Schema migration and Full data migration.

    Note The task automatically stops when its schedule ends. To stop the task before the scheduled end time, click Stop data integration task.