ApsaraDB for ClickHouse:Migrate data between ApsaraDB for ClickHouse clusters

Prerequisites

• Both the source and destination clusters are ApsaraDB for ClickHouse clusters.

• Usernames and passwords of database accounts are created for the source and destination clusters.

• The replicas of the source and the destination clusters must be consistent.

• The source and destination clusters must be deployed in the same region and use the same virtual private cloud (VPC).

• If you enable tiered storage of hot data and cold data for the source cluster, you must also enable tiered storage of hot data and cold data for the destination cluster. If you disable tiered storage of hot data and cold data for the source cluster, you must also disable tiered storage of hot data and cold data for the destination cluster.

• Each local table in the source cluster must correspond to a unique distributed table.

• The storage space of the destination cluster must be greater than or equal to 1.2 times the storage space of the source cluster.

• The IP address of your source ApsaraDB for ClickHouse cluster is added to the IP address whitelist of your destination ApsaraDB for ClickHouse cluster.

• The IP address of your destination ApsaraDB for ClickHouse cluster is added to the IP address whitelist of your source ApsaraDB for ClickHouse cluster.

Precautions

• You cannot migrate data from a cluster of a later version to a cluster of an earlier version.

• Before you migrate data, make sure that no management operations, such as scale-outs, upgrades, or downgrades, are being performed on the source and destination clusters.

• Full data migration and incremental data migration are supported between ApsaraDB for ClickHouse clusters.

• For non-MergeTree tables such as external tables and log tables, only the schemas are migrated. The data is not migrated. Therefore, if the source cluster contains non-MergeTree tables and you migrate data from the source cluster to the destination cluster, the non-MergeTree tables in the destination cluster have only a table schema and no specific business data.

Impacts of data migration

• During the migration, you can read data from or write data to the source cluster of V20.8 or later. To ensure that the migration task is completed in a timely manner, we recommend that you stop writing data to the source cluster when 95% of the data has been migrated. For the source cluster of V20.3 or earlier, you can read data but cannot write data until the migration is complete.

• In addition, you cannot read data from or write data to the destination cluster until the migration is complete.

Supported versions

The following table describes the version mappings between source clusters and destination clusters.

Procedure

1. Log on to the ApsaraDB for ClickHouse console.

2. On the Clusters page, click the Default Clusters tab, and click the ID of the destination cluster.

3. In the left-side navigation pane of the page that appears, click Migrate Instance.

4. On the Cluster page that appears, click Create Migration Task.

   

   Note

   If no data is migrated to the selected cluster, the message "No data found." is displayed.

5. In the Select the source and destination clusters step, select a source cluster ID from the Instance ID drop-down list, and configure the Database Account and Database Password parameters of the source and destination clusters.

   

6. Click Test Connectivity and Proceed to check whether the two clusters are connected.

   • If the connection is established, proceed to the next step.

   • If the connection fails to be established: If the message Source cluster account or password error. appears, modify the Database Account or Database Password parameters.

7. After the source and destination clusters are connected, proceed to the Migration Content step. Click Next: Pre-detect and Start Synchronization.

   

   Note

   You can migrate the following objects from the source cluster: the cluster, databases, tables, data dictionaries, materialized views, user permissions, and cluster configurations.

8. After the migration task is configured, the system pre-checks the migration configuration and starts the migration task in the backend.

   • If no issues are detected during the precheck, click Completed.

   • If an issue is detected, follow the on-screen instructions to resolve the issue, and then migrate the data again. The following table describes the precheck items.

     Item	Description
     Cluster Status Detection	Before you migrate data, make sure that no management operations, such as scale-outs, upgrades, or downgrades, are being performed on the source and destination clusters. If management operations are being performed on the source and destination clusters, the system cannot start a migration task.
     Storage Space Detection	Before the system starts a migration task, the system checks the storage capacity of the source and destination clusters. Make sure that the storage space of the destination cluster is greater than or equal to 1.2 times the storage space of the source cluster.
     Local Table and Distributed Table Detection	If no distributed table is created for a local table or multiple distributed tables are created for the same local table, the precheck fails. You must delete redundant distributed tables to ensure that the distributed table of each local table is unique.

9. After the migration task is complete, you can view the migration task on the Migrate Cluster page.

   

   Note

   If Migration Status is Completed, data is migrated from the source cluster to the destination cluster.

References

For information about how to migrate data from a self-managed ClickHouse cluster to an ApsaraDB for ClickHouse cluster, see Migrate data from a self-managed ClickHouse cluster to an ApsaraDB for ClickHouse cluster.