This topic describes how to upgrade the major engine version of an ApsaraDB for ClickHouse Community-compatible Edition cluster with one click in the console.
Background
To improve your service experience, we recommend that you promptly upgrade older clusters to the latest version. ApsaraDB for ClickHouse Community-compatible Edition provides a one-click upgrade feature in the console. The upgrade process varies based on the cluster architecture:
Old-architecture clusters: The system automatically creates a new cluster that runs the latest version and migrates data from the source cluster to the new cluster to complete the upgrade.
New-architecture clusters: The system directly updates the instance version by cloning the instance and upgrading the kernel.
To ensure a smooth transition, select an upgrade operation based on your cluster version.
Preparations: Confirm the cluster architecture
Before you upgrade, confirm the cluster architecture in the console to select the appropriate validation method and procedure. To confirm the cluster architecture, perform the following steps:
On the Clusters page, click the ID of the target cluster to go to the Cluster Information page.
In the Cluster Properties section, click Major Version Upgrade to the right of Version.
In the Major Version Upgrade dialog box, view the third configuration item for setting the time.
Version Upgrade Execution Time: Indicates a new-architecture cluster.
Write Suspension Time: Indicates an old-architecture cluster.
To upgrade a new-architecture cluster, see Upgrade a new-architecture cluster.
To upgrade an old-architecture cluster, see Upgrade an old-architecture cluster.
Upgrade a new-architecture cluster
Prerequisites
The cluster is in the Running state.
Usage notes
After the one-click major engine version upgrade process starts, it cannot be canceled.
NoteVersion rollback is not supported after a one-click major engine version upgrade. If you need the ability to roll back to the previous version after the upgrade, you can upgrade by cloning the cluster. For more information, see Upgrade the major engine version by cloning.
A one-click major engine version upgrade involves only one cluster, and you cannot roll back to the source version after the upgrade.
To ensure a smooth upgrade, stop write operations from your business before you start the upgrade. During the upgrade, the cluster automatically stops writes. However, if your business does not stop writing in advance, the synchronization wait time may be excessively long, or synchronization may not complete in a timely manner. This can affect the timeliness of the upgrade.
To reduce the impact of the upgrade on your business, perform the following validations before you start the upgrade.
Compatibility and performance validation: Validate differences in the features, syntax, and performance of write and query operations.
Upgrade duration validation: The instance restarts multiple times during the version upgrade. Some instances may take a long time to restart because of many databases, tables, or partitions (Parts).
ImportantIf the cluster has a large amount of data in cold storage and you use clone validation, the one-click upgrade takes longer than the clone validation. This is because the clone operation does not include cold data.
For more information about how to perform the validation, see Validate version compatibility.
Impact on the cluster
During the version upgrade, the cluster restarts multiple times and is unavailable for read and write operations. We recommend that you perform the upgrade during off-peak hours.
Validate version compatibility
Because a one-click major engine version upgrade involves only one cluster and cannot be rolled back, you must test the upgrade first. Before you proceed, confirm that the new version is fully compatible with the features of the source version and verify the upgrade duration. The following steps describe the procedure:
If Tiered Storage for Hot and Cold Data is enabled for the cluster, the clone operation includes only hot data. You cannot query cold data in the new cluster.
Go to the Cluster Information page of the target instance. In the navigation pane on the left, click Backup And Recovery. After the instance is backed up, click Restore Instance.
Select to clone from a real-time replica and set the destination kernel version to the target upgrade version.
Create the cloned cluster.
Perform compatibility validation.
Validate the compatibility of business queries with the new version. For more information, see SQL compatibility validation.
Perform regression testing on business features.
Procedure
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters, and then click Clusters of Community-compatible Edition tab.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Properties section, click Major Version Upgrade to the right of Version.
Configure the following parameters as prompted and click the OK button.
Configuration item
Description
Example
Upgrade Cluster Kernel Version to
The target cluster version. The version cannot be rolled back after the upgrade.
Currently, only version 25.3 is supported.
25.3 (LTS Version)
Version Upgrade Execution Time
The execution time for the cluster version upgrade.
ImportantAfter you select a specified time or upgrade within a maintenance window, the cluster status changes from Running to Upgrading. Before the specified time or maintenance window arrives, the cluster can serve read and write requests normally, but you cannot perform O&M operations such as upgrades, downgrades, scaling, or migrations.
Specified Time: Select a future point in time to perform the major version upgrade.
Upgrade Within Maintenance Window: The maintenance window of the current cluster is selected by default.
Update Now: Execute the upgrade operation immediately.
2024-05-29 14:46
Whether to Perform Clone Validation
Select Clone Validation Performed or Skip Clone Verification (Not Recommended).
Clone Validation Performed
Upgrade an old-architecture cluster
Prerequisites
The cluster is in the Running state.
Usage notes
A one-click major engine version upgrade involves only one cluster, and you cannot roll back to the source version after the upgrade.
NoteVersion rollback is not supported after a one-click major engine version upgrade. If you need the ability to roll back to the previous version after the upgrade, you can upgrade by migrating data. For more information, see Upgrade by migration.
When you upgrade the cluster, note the following points about the databases and tables:
For tables that use a MergeTree-family engine, historical data is migrated to the new cluster and automatically redistributed during the upgrade.
For tables that do not use a MergeTree-family engine, such as external tables or Log tables, only the table schemas are migrated. Data is not migrated.
For materialized views, only the schemas are migrated. Data is not migrated.
Tables that use the Kafka or RabbitMQ engine are not supported for migration. You must delete these tables before the upgrade.
After the upgrade, the IP addresses of the inner nodes change. If data writing and access depend on node IP addresses, you must obtain the VPC CIDR block of the cluster again. For more information, see Obtain the VPC CIDR block of a cluster.
To reduce the impact of the upgrade on your business, perform the following validations before you start the upgrade.
Compatibility and performance validation: Validate differences in the features, syntax, and performance of write and query operations.
Upgrade duration validation: The instance restarts multiple times during the version upgrade. Some instances may take a long time to restart because of many databases, tables, or partitions (Parts).
For more information about how to perform the validation, see Validate version compatibility.
Impact on the cluster
During the upgrade of a Community-compatible Edition cluster, the cluster is available for read and write operations, except for the last 10 minutes of migration when writes are not allowed. To view the remaining migration time, see View the upgrade progress.
Validate version compatibility
Because a one-click major engine version upgrade involves only one cluster and cannot be rolled back, you must test the upgrade first. Before you proceed, confirm that the new version is fully compatible with the features of the source version and verify the upgrade duration. The following steps describe the procedure:
Purchase a new cluster to perform migration validation. For more information, see Upgrade the major engine version by data migration.
In the new cluster, perform SQL compatibility validation. For more information, see SQL compatibility validation.
Perform regression testing on business features to validate them.
Upgrade the cluster
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters, and then click Clusters of Community-compatible Edition tab.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Properties section, click Major Version Upgrade to the right of Version.
Configure the following parameters as prompted and click the OK button.
Configuration item
Description
Example
Upgrade Instance Kernel Version To
The target cluster version. The version cannot be rolled back after the upgrade.
Currently, only version 23.8 is supported.
23.8 (LTS Version)
Write Suspension Time
To ensure data consistency between the pre-upgrade and post-upgrade states, the cluster automatically stops write operations during the last 10 minutes of the upgrade. The following rules apply to setting the write suspension time:
To ensure a successful upgrade, set the write suspension time to at least 30 minutes.
The upgrade must finish within 5 days. Therefore, the end date of the Write Suspension Time must be less than or equal to
current date + 5 days.To reduce the impact of write suspension on your business, set the write suspension time to a period during your off-peak hours.
2025-03-20 10:08-2025-03-25 10:08Perform Instance Migration Validation
Select Instance Migration Validation Performed or Skip Instance Migration Validation (not Recommended).
Instance migration validation performed
Next steps
To view the upgrade progress, see View the upgrade progress.
If the specified write suspension time has passed but the cluster status is still Upgrading, you must modify the write suspension time to complete the migration.
If the upgrade affects your business and you want to stop it quickly, you can cancel the upgrade.
View the upgrade progress
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters, and then click Clusters of Community-compatible Edition tab.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Status section, click View Version Upgrade Progress to the right of Status.
In the Modify Write Suspension Window dialog box that appears, you can view the upgrade progress. For example, you can view the progress of MergeTree schema migration, data migration, the estimated remaining time for data migration, and the progress of other schema migrations.
Modify the write suspension time
If the specified write suspension time has passed but the cluster status is still Upgrading, the data migration is not yet complete. You must adjust the write suspension time to ensure that the upgrade finishes successfully.
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters, and then click Clusters of Community-compatible Edition tab.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Status section, click Version to the right of View Version Upgrade Progress.
In the Modify Write Suspension Window dialog box that appears, modify the Write Suspension Time and click Confirm.
NoteThe rules for setting the Write Suspension Time are the same as those for the Write Suspension Time parameter described in Upgrade the cluster.
Cancel the upgrade
If the upgrade affects your business and you want to stop it quickly, you can cancel the upgrade to terminate the operation.
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters, and then click Clusters of Community-compatible Edition tab.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Status section, click View Version Upgrade Progress to the right of Version.
In the Modify Write Suspension Window dialog box that appears, click Cancel Upgrade.
NoteAfter you click Cancel Upgrade, the upgrade task does not stop immediately. The task stops completely after about 5 minutes.
FAQ
Q: What do I do if the error "Unsupported Kafka table definition" is reported during a major engine version upgrade?
A: Kafka tables in the current version do not support using the DEFAULT keyword to define default field values. This causes the kernel package to fail to start. To resolve this issue, perform the following steps:
Run the
select create_table_query from system.tables where engine = 'Kafka'statement to find all Kafka tables.Back up the Data Definition Language (DDL) statements of the tables that you found.
Delete the tables that you found.
Re-create the tables.
ImportantWhen you re-create the tables, do not use the
DEFAULTkeyword to define default field values.
Q: What do I do if the error "Unsupported MaterializedMySQL table definition" is reported during a major engine version upgrade?
A: The configuration parameters of the MaterializedMySQL engine in the current version are incompatible with those in the source cluster's version. To resolve this issue, perform the following steps:
Run the
select name from system.databases where engine = 'MaterializedMySQL'statement to find the databases that use the MaterializedMySQL engine.Back up the DDL statements of the databases that you found.
Delete the databases that you found.
Upgrade the kernel version.
Adjust the backed-up DDL statements to be compatible with the target version, and then re-create the databases that use the MaterializedMySQL engine.
Q: What do I do if the error "Unsupported table definition other than 20.3: Nullable(Array(*))/SecondaryIndex(KEY definition exists)" is reported during a major engine version upgrade?
A: If your cluster version is 20.3, you might be using some features developed by Alibaba Cloud, such as:
Defining a table field of the Nullable(Array(*)) type.
A secondary index defined using the KEY keyword.
Because these features have not been merged back into open-source ClickHouse, instance versions later than 20.8 do not include them. You must adjust the relevant tables before you upgrade. You can perform the following operations:
Validate the compatibility between the source cluster version and the target version.
ImportantBecause the version span between 20.3 and the current version is large, you must perform a thorough validation before you implement the upgrade to avoid affecting your business.
Delete the field modified with
Nullable(Array(*))and then add the field again.Delete the secondary index that is defined with the KEY keyword. After the upgrade is complete, add a skipping index to the table again.
ImportantThe two types of indexes have different implementation principles, which may cause performance differences.
References
Data migration and synchronization
Upgrade the major engine version by data migration