All Products
Search
Document Center

Evaluate data import from RDS database to DRDS

Last Updated: Aug 15, 2020

If you use ApsaraDB for RDS (RDS), you will encounter performance bottlenecks as the data volume, storage, concurrency, and queries per second (QPS) increase with businesses growth. You can migrate data from RDS to Distributed Relational Database Service (DRDS) for database sharding and table sharding.

You may face the following challenges during the migration:

  • How do I select the DRDS instance type?
  • How do I select the type of RDS instances attached to the DRDS instance?
  • How do I partition a single table after the table is migrated from the RDS instance to the DRDS instance? How do I select the number of shards and the shard keys?
  • How can I create databases and tables quickly?
  • How can I import data from the RDS instance to the DRDS instance quickly?

The evaluation and import feature is introduced to solve the preceding problems. It provides technical solutions that allow you to quickly switch from RDS to DRDS.

Advantages

  • The evaluation and recommendation feature provided for migrating data from an RDS instance to a DRDS instance in this solution can:
    • Provide recommendations on the types of the destination DRDS instance and its attached RDS instances.
    • Provide sharding solutions for each source RDS table.
  • The data import feature provided for migrating data from an RDS instance to a DRDS instance in this solution supports:
    • Evaluation-based import: Based on the evaluation and recommendations, an integrated migration solution is provided to quickly create databases, create tables, and import data.
    • Custom import: Users who are experienced in using DRDS or are familiar with database sharding can purchase DRDS or RDS instances based on their solutions. After creating databases and tables, they can import data from the source RDS database to the destination DRDS database as required.

Functional modules

  • Evaluation and recommendation
  • Data import

Evaluation and recommendation

  • Description
    The evaluation and recommendation module performs evaluation based on your source RDS instance, and provides recommendations for migrating data to the DRDS instance. All the recommendations are displayed in an evaluation report. The evaluation report includes:

    • Type of the destination DRDS instance
    • Type of the RDS instances attached to the DRDS instance after the migration
    • The sharding recommendations include:
      • Whether to partition each table in the source database, and how to select the shard key.
      • The table creation statements used when tables in the source database are imported to DRDS. All table creation statements can be exported.
  • Procedure

    1. Start DRDS evaluation.
      1
    2. Fill in the information of the source RDS instance and the source database.
      2
    3. Perform precheck and start task.
      3
  • Evaluation report example5

  • Requirements

    • You must provide the names of the source RDS instance and the source database.
    • You must provide a read account that can access the source RDS instance.
    • You must allow the activation of non-beta SQL Explorer for RDS.
    • You must enter an estimated increase percentage of the data size within the next year.
    • You must enter the estimated increase percentage of the queries per second (QPS) within the next year.
  • Data pulling
    The following data must be pulled to provide evaluation and recommendations:

    • Table creation statements of all source tables.
    • The data volume of all tables obtained by running INFORMATION SCHEMA in the source database.
    • The disk usage of the source RDS instance, including the peak IOPS in the last week and the peak QPS in the last week.
    • The SQL logs of the source RDS instance.
  • How to perform evaluation

    • The module recommends the DRDS instance type based on your peak QPS in the last week and the estimated QPS increase in the next year.
    • The module recommends the type of the RDS instance attached to the DRDS instance based on your disk usage of the RDS instance, the peak IOPS in the last week, and estimated data increase in the next year.
    • The module determines whether to partition tables and the number of table shards based on the data volume of each table.
    • The module recommends the shard keys for each table by analyzing SQL logs.

Data import

The data import module allows you to import data from an RDS instance to a DRDS instance by using either of the following methods:

  • Evaluation-based import
  • Custom import

Evaluation-based import

  • Description
    Evaluation-based import refers to one-click import based on an evaluation report. During the import process, you need to:

    • Purchase a DRDS instance.
    • Purchase RDS instances.
    • You can also use existing DRDS and RDS instances. However, we recommend that you purchase new ones.
  • Procedure
    In the Assessment Recommendations page, find the target task, click Start Importing on the Operations column and follow the instructions to start a importing task.
    6

  • Task import process1

Custom import

  • Description
    In addition to evaluation-based import, DRDS also provides custom import for users who are experienced in using DRDS or familiar with database sharding.
    To use custom import, you must first:

    • Purchase the destination DRDS instance and the RDS instances attached to the DRDS instance.
    • Create a destination database on the destination DRDS instance and create tables in the database.
  • Procedure

    1. On the DRDS console, click on the Data Import on the left navigation pane.
    2. In the Import Task page, click on RDS Custom Import.
      1
    3. Fill in the information of the source and destination databases.
      1
    4. Determine the tables to be migrated.
    5. Precheck and start the import task.
      1
  • Requirements

    • You have created a DRDS instance and created databases and tables.
    • You must provide the names of the source RDS instance and source database, and a read account that can access the source database.
    • You must provide the names of the destination DRDS instance and destination database name, and a read/write account that can access the destination database.
  • Task execution process2

Business switchover

After the full synchronization task is completed, DRDS provides another 14 days of incremental synchronization.
After the data import is completed, you can switch your businesses from the RDS instance to the DRDS instance as follows:

  • Click End Task on the Import Task page. The incremental synchronization service stops.
  • Switch your businesses from the RDS instance to the DRDS instance.