ApsaraDB for SelectDB:Data migration

Feature overview

The data migration feature supports full data migration from the following data sources to ApsaraDB for SelectDB:

• Apache Doris

• SelectDB

• StarRocks

Prerequisites

• You have prepared the source cluster connection information, including the address, port, username, and password.

• Prepare a temporary Object Storage Service (OSS) bucket for data staging. Retrieve its endpoint and authorization information. The migration task uses this OSS bucket to temporarily store data and automatically cleans up the data after the task completes.

Procedure

Navigate to the Data Migration page

1. Log on to the ApsaraDB for SelectDB console.

2. In the upper-left corner of the page, select the region where your instance resides.

3. On the Instances page, click the target Instance ID to go to the Instance Details page.

4. Click Data Development and Management (Studio) in the left-side menu of the page, and then click data migration.

   Note

   The first time you use Data Development and Management, the system prompts you to add your local public IP address to the webui_whitelist IP address whitelist group. Read the prompt carefully and click OK.

Create a migration task

On the Data Migration page, click Create Migration Task and follow the wizard to complete the configuration step by step.

1. Configure the source cluster and basic task information

   In the navigation pane on the left, go to the migration module and click Create Migration Task. Enter the required information as prompted, then click Next to verify connectivity.

   • Task Name and Description: Enter a custom name and description for the task.

   • Source Cluster Connection Information: Enter the endpoint, JDBC port, HTTP port, username, and password for the source cluster.

2. Configure the temporary object storage

   Select the cloud service provider for the object storage, then enter the corresponding Object Storage Endpoint and Authorization Information.

3. Select objects to migrate

   Select the databases and tables to migrate. You can select multiple objects at once or use the search box to filter them. You can also use the Data Filtering feature to set filter conditions for specific tables and migrate only data that meets those conditions. The migration task automatically creates corresponding tables in the destination ApsaraDB for SelectDB based on the table schema of the source, so you do not need to create the table in advance. If you connect to ApsaraDB for SelectDB over JDBC (MySQL protocol) and execute an INSERT statement to write data, a Table does not exist error is returned when the destination table does not exist. In this case, you must execute CREATE TABLE to create the table schema before you write data.

4. Advanced settings

   Configure performance metrics for the migration task:

   • Cluster: Select the compute resource to run the migration task.

   • Table-level concurrency: Set the number of tables to migrate simultaneously.

   • In-table concurrency: Set the number of concurrent data partitions (Tablets) to migrate within a single table.

5. Precheck and start

   The system performs a precheck on the configuration. After the precheck passes, start the migration task.

Data consistency

Whether the table schema and data remain consistent after migration depends on the field type:

• Regular fields, such as INT, VARCHAR, and DECIMAL, retain a consistent table schema and data.

• BITMAP and HLL aggregate fields do not support direct migration. You must re-create the table on the destination and re-import the data by using the corresponding functions:

  • BITMAP fields: Specify the BITMAP_UNION aggregate type when you create the table, and use the bitmap_hash() function to import data.

  • HLL fields: Specify the HLL_UNION aggregate type when you create the table, and use the hll_hash() function to import data.

After the migration is complete, compare the table schema (column names, types, and constraints) and data integrity (row count and data content) between the source and the destination to confirm that the migration result meets your expectations.

View and manage migration tasks

After you start a migration task, the page automatically redirects to the Data Migration list. You can view and manage all migration tasks here.

• View the task list: On the Data Migration page, view basic information for all tasks, such as Name, Data Source, Status, and Created At.

• View task details: Click the button in the Actions column for a task to open the task details page. There, you can review the migration progress, table-level details, and configuration information.