PolarDB:Migrate data from RDS to PolarDB-X 1.0

Background

When you use the RDS service, the data volume, storage, concurrency, and queries per second (QPS) are increased along with the business growth. This may cause performance bottlenecks. You can migrate data from RDS to PolarDB-X 1.0 for database sharding and table sharding. You may need to fix the following issues during the data migration from RDS to PolarDB-X 1.0:

• How do I select the PolarDB-X 1.0 instance type?
• How do I select the type of RDS instance that is attached to the PolarDB-X 1.0 instance?
• How do I shard a RDS table after the table is migrated to PolarDB-X 1.0? How do I determine the number of shards and the shard keys?
• How can I create databases and tables?
• How can I quickly import data from RDS to PolarDB-X 1.0?

Solutions

To fix these issues, PolarDB-X 1.0 provides the evaluation import feature.PolarDB-X 1.0 allows you to simplify data migration from RDS to PolarDB-X 1.0.

Step 1: Obtain evaluation suggestions

PolarDB-X 1.0 evaluates the source RDS and provides suggestions on data migration to PolarDB-X 1.0. All suggestions are included in the evaluation report.

1. Log on to the PolarDB for Xscale console.
2. In the top navigation bar, select the region where the target instance is located.
3. In the left-side navigation pane, choose Evaluation import > Suggestions.
4. In the upper-right corner of the Suggestions page, click Start RDS assessment.
   
5. On the SOURCE database information page, set the following parameters.

   Parameter		Description
   Task name		The name of a data migration task.
   Source Database Information	Instance type	Use the default value of RDS instance.
   	RDS instance ID	Select the source RDS instance.
   	Database name	Select the source database.
   	Database port	Enter the port number of the source database.
   	Database account	Enter the account of the source database.
   	Database password	Enter the password of the source database account. Note After you enter the account and password of the source database, click Test connection to check whether the source RDS can be connected.
   Business Information		You can set the Expected data scale growth percentage and Expected average QPS growth percentage to proper values based on your business requirements to obtain more accurate assessment suggestions.
   Evaluation duration		You can set the evaluation duration to 1 hour, 3 hours, 6 hours, 1 day, 3 days, or 7 days. You can also enter a custom evaluation duration from 1 to 336 hours (14 days).

6. Select Agree to open SQL insight and click Next.
7. On the Task preview page, click Start task when the Precheck Result displays Success.
8. After the assessment task is completed, perform the following steps:
   

   • You can click Assessment Report in the Operation column to view evaluation details.
     Note You can click Show in the Split Recommendation section to view details of sharding, such as the sharding type and shard key. You can also click Export and create table SQL to download the SQL statements that are used to create tables.
   • You can click Start Import in the Operation column to import data.

Step 2: Import data

You can import data from the source RDS to PolarDB-X 1.0 in the following ways:

• Evaluation-based import:
  1. Log on to the PolarDB-X 1.0 console.
  2. In the top menu bar, select the region where the destination instance is deployed.
  3. In the left-side navigation pane, choose Evaluation import > Suggestions.
  4. On the page that appears, find the task and click Start Import in the Operation column.
  5. On the Assessment Summary page, confirm the recommended instance type and number, and click Next.
  6. Select the PolarDB-X 1.0 instance, click to add the instance to the selected section on the right, and then click Next.
     Note You can select only one PolarDB-X 1.0 instance.
  7. Select the ApsaraDB RDS for MySQL instance, click to add the instance to the selected section on the right, and then click Next.
     Note You can continue to use the purchased ApsaraDB RDS for MySQL instance. However, we recommend that you purchase a new ApsaraDB RDS for MySQL instance as a separate storage resource for PolarDB-X 1.0.
  8. On the Task preview page, click Next when the Precheck Result displays Success.
  9. On the Topology confirmation page, confirm the source RDS instance, source database, and the destination PolarDB-X 1.0 instance, and click Create task.
• Custom import
  1. Log on to the DRDS console.
  2. In the top menu bar, select the region where the destination instance is deployed.
  3. In the left-side navigation pane, choose Evaluation import > Data import.
  4. In the upper-right corner of the Import tasks page, click RDS custom import.
  5. On the Source database and target database page, configure the following parameters and click Next.

     Parameter		Description
     Task name		The name of a data migration task.
     Source Database Information	Instance type	Use the default value of RDS instance.
     	RDS instance ID	Select the source RDS instance.
     	Database name	Select the source database.
     	Database port	Enter the port number of the source database.
     	Database account	Enter the account of the source database.
     	Database password	Enter the password of the source database account. Note After you enter the account and password of the source database, click Test connection to check whether the source RDS can be connected.
     Target database information	Database port	Enter the port number of the destination database.
     	Database account	Enter the account of the destination database.
     	Database password	Enter the password of the destination database account. Note After you enter the account and password of the destination database, click Test connection to check whether the destination PolarDB-X 1.0 can be connected.

  6. On the Migration list page, select the source table from the SOURCE table name column on the left, select the destination table from Target table name on the right, and then click Next.
     Important Tables that use a composite primary key cannot be migrated. Otherwise, an error occurs in the Task preview step.
  7. On the Task preview page, click Start task when the Precheck Result displays Success.

Step 3: Complete the data migration

1. Log on to the PolarDB for Xscale console.
2. In the top navigation bar, select the region where the target instance is located.
3. In the left-side navigation pane, choose Evaluation import > Data import.
4. On the page that appears, find the destination table and click End Task in the Operation column.
   

   Note

   • After the full data is migrated, PolarDB-X 1.0 provides a 14-day synchronization service for incremental data. After the Task progress is changed to 100%, you can click End Task to complete data migration. You can also wait until the system stops 14-day synchronization for incremental data and automatically End Task.
   • The Task Status of the data import is changed from Running to Ended after End Task. You can click Cleanup Tasks to delete the task. Deleted tasks are not displayed in the task list.

FAQ

Q: Do I need to stop excuting writing data to the source database before data importing?

A: No. After the full data is migrated, PolarDB-X 1.0 provides a 14-day synchronization service for incremental data. After the Task progress is changed to 100%, you can click End Task to complete data migration. You can also wait until the system stops 14-day synchronization for incremental data and automatically End Task.