This topic describes how to upgrade the major engine version of an ApsaraDB for ClickHouse Community-compatible Edition cluster with one click in the console.
Background
To improve your service experience, promptly upgrade older clusters to the latest version. ApsaraDB for ClickHouse Community-compatible Edition provides a one-click upgrade feature in the console. The upgrade process varies based on the cluster architecture:
For old-architecture clusters, the system automatically creates a new cluster that runs the latest version and migrates data from the source cluster to the new cluster to complete the upgrade.
For new-architecture clusters, the system directly updates the instance version by cloning the instance and upgrading the kernel.
Choose the appropriate upgrade operation for your cluster version to ensure a smooth transition.
Preparations: Confirm the cluster architecture
Before you upgrade, confirm the cluster architecture in the console to select the appropriate validation method and procedure. The procedure is as follows:
On the Clusters page, click the ID of the target cluster to go to the Cluster Information page.
In the Cluster Properties section, click Major Version Upgrade to the right of Version.
In the Major Version Upgrade dialog box, view the third configuration item for setting the time.
Version Upgrade Execution Time:: Indicates a new-architecture cluster.
Time of Stopping Data Writing: Indicates an old-architecture cluster.
To upgrade a new-architecture cluster, see Upgrade a new-architecture cluster.
To upgrade an old-architecture cluster, see Upgrade an old-architecture cluster.
Upgrade a new-architecture cluster
Prerequisites
The cluster is in the Running state.
Usage notes
After the one-click major engine version upgrade process starts, it cannot be canceled.
NoteVersion rollback is not supported after a one-click major engine version upgrade. If you need the ability to roll back to the previous version after the upgrade, you can upgrade by cloning the cluster. For more information, see Upgrade the major engine version by cloning.
The one-click major engine version upgrade applies to a single cluster, and you cannot roll back to the source version after the upgrade is complete.
To ensure a smooth upgrade, stop write operations from your application before you start the upgrade. The cluster automatically stops writes during the upgrade. However, if your application does not stop writing in advance, the synchronization wait time may be excessively long or synchronization may not complete promptly. This can affect the timeliness of the upgrade.
To reduce the impact of the upgrade on your services, perform the following validations before you start the upgrade.
Compatibility and performance validation: Validate differences in the features, syntax, and performance of write and query operations.
Upgrade duration validation: The instance restarts multiple times during the version upgrade. Some instances may take a long time to restart because they have many databases, tables, or partitions (Parts).
ImportantIf the cluster has a large amount of data in cold storage and you use clone validation, the one-click upgrade takes longer than the clone validation. This is because the clone operation does not include cold data.
For more information about how to perform the validation, see Validate version compatibility.
Impact on the cluster
During the version upgrade, the cluster restarts multiple times and is unavailable for read and write operations. Perform the upgrade during off-peak hours.
Validate version compatibility
Because a one-click major engine version upgrade involves only one cluster and cannot be rolled back, you must test the upgrade first. Before you proceed, confirm that the new version is fully compatible with the features of the source version and verify the upgrade duration. The following steps describe the procedure:
If Tiered Storage of Hot and Cold Data is enabled for the cluster, the clone operation includes only hot data. You cannot query cold data in the new cluster.
Go to the Cluster Information page of the target instance. In the navigation pane on the left, click Backup and Restoration. After the instance is backed up, click Restore Instance.
Select to clone from a real-time replica and set the destination kernel version to the target upgrade version.
Create the cloned cluster.
Perform compatibility validation.
Validate the compatibility of your application queries with the new version. For more information, see SQL compatibility validation.
Let's go back to the business features.
Procedure
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters of Community-compatible Edition.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Properties section, click Major Version Upgrade to the right of Version.
Configure the following parameters as prompted and click the OK button.
Configuration item
Description
Example
Upgrade Cluster Kernel Version to
The target cluster version. The version cannot be rolled back after the upgrade.
Currently, only version 25.3 is supported.
23.8 (LTS Version)
Version Upgrade Execution Time
The execution time for the cluster version upgrade.
ImportantAfter you select a specified time or upgrade within a maintenance window, the cluster status changes from Running to Upgrading. Before the specified time or maintenance window arrives, the cluster can serve read and write requests normally, but you cannot perform O&M operations such as upgrades, downgrades, scaling, or migrations.
Specified Time: Select a future point in time to perform the major version upgrade.
Upgrade Within Maintenance Window: The maintenance window of the current cluster is selected by default.
Update Now: Execute the upgrade operation immediately.
2024-05-29 14:46
Whether to Perform Clone Validation
Select Clone Validation Performed or Skip Clone Verification (Not Recommended).
Clone Validation Performed
Upgrade an old-architecture cluster
Prerequisites
The cluster is in the Running state.
Usage notes
The one-click major engine version upgrade applies to a single cluster, and you cannot roll back to the source version after the upgrade is complete.
NoteVersion rollback is not supported after a one-click major engine version upgrade. If you need the ability to roll back to the previous version after the upgrade, you can upgrade by migrating data. For more information, see Upgrade by migration.
When you upgrade the cluster, note the following points about the databases and tables:
For tables that use a MergeTree-family engine, historical data is migrated to the new cluster and automatically redistributed during the upgrade.
For tables that do not use a MergeTree-family engine, such as external tables or Log tables, only the table schemas are migrated. Data is not migrated.
For materialized views, only the schemas are migrated. Data is not migrated.
Tables that use the Kafka or RabbitMQ engine are not supported for migration. You must delete these tables before the upgrade.
After the upgrade, the IP addresses of the inner nodes change. If data writing and access depend on node IP addresses, you must retrieve the VPC CIDR block of the cluster again. For more information, see Obtain the VPC CIDR block of a cluster.
To reduce the impact of the upgrade on your services, perform the following validations before you start the upgrade.
Compatibility and performance validation: Validate differences in the features, syntax, and performance of write and query operations.
Upgrade duration validation: The instance restarts multiple times during the version upgrade. Some instances may take a long time to restart because they have many databases, tables, or partitions (Parts).
For more information about how to perform the validation, see Validate version compatibility.
Impact on the cluster
During the upgrade of a Community-compatible Edition cluster, the cluster is available for read and write operations, except for the last 10 minutes of migration when writes are not allowed. To view the remaining migration time, see View the upgrade progress.
Validate version compatibility
Because a one-click major engine version upgrade involves only one cluster and cannot be rolled back, you must test the upgrade first. Before you proceed, confirm that the new version is fully compatible with the features of the source version and verify the upgrade duration. The following steps describe the procedure:
Purchase a new cluster to perform migration validation. For more information, see Upgrade the major engine version by data migration.
In the new cluster, perform SQL compatibility validation. For more information, see SQL compatibility validation.
Perform regression testing on your application features to validate them.
Upgrade the cluster
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters of Community-compatible Edition.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Properties section, click Major Version Upgrade to the right of Version.
Configure the following parameters as prompted and click the OK button.
Configuration item
Description
Example
Upgrade Instance Kernel Version To
The target cluster version. The version cannot be rolled back after the upgrade.
Currently, only version 23.8 is supported.
23.8 (LTS Version)
Time of Stopping Data Writing
To ensure data consistency between the pre-upgrade and post-upgrade states, the cluster automatically stops write operations during the last 10 minutes of the upgrade. The following rules apply to setting the write suspension time:
To ensure a successful upgrade, set the write suspension time to at least 30 minutes.
The upgrade must finish within 5 days. Therefore, the end date of the Time of Stopping Data Writing must be less than or equal to
current date + 5 days.To reduce the impact of write suspension on your business, set the write suspension time to a period during your off-peak hours.
2025-03-20 10:08-2025-03-25 10:08Perform Instance Migration Validation
Select Verification Performed or Skip Instance Migration Validation (not Recommended).
Instance migration validation performed
Next steps:
To view the upgrade progress, see View the upgrade progress.
If the specified write suspension time has passed but the cluster status is still Upgrading, you must modify the write suspension time to complete the migration.
If the upgrade affects your services and you want to stop it, you can cancel the upgrade.
View the upgrade progress
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters of Community-compatible Edition.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Status section, click View Progress to the right of Status.
In the Modify Data Write-Stop Time Window dialog box that appears, you can view the upgrade progress. For example, you can view the progress of MergeTree schema migration, data migration, the estimated remaining time for data migration, and the progress of other schema migrations.
Modify the write suspension time
If the specified write suspension time has passed but the cluster status is still Upgrading, the data migration is not yet complete. You must adjust the write suspension time to ensure that the upgrade finishes successfully.
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters of Community-compatible Edition.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Status section, click View Progress to the right of Status.
In the Modify Data Write-Stop Time Window dialog box that appears, modify the Time of Stopping Data Writing and click Confirm.
NoteThe rules for setting the Time of Stopping Data Writing are the same as those for the Time of Stopping Data Writing parameter described in Upgrade the cluster.
Cancel the upgrade
If the upgrade affects your services and you want to stop it, you can cancel the upgrade.
Log on to the ApsaraDB for ClickHouse console with your Alibaba Cloud account.
In the upper-left corner of the page, select the region where the target cluster is located.
In the navigation pane on the left, click Clusters of Community-compatible Edition.
Find the target cluster and click the cluster ID to go to the Cluster Information page.
In the Cluster Status section, click View Progress to the right of Status.
In the Modify Data Write-Stop Time Window dialog box that appears, click Cancel Upgrade.
NoteAfter you click Cancel Upgrade, the upgrade task does not stop immediately. The task stops completely after about 5 minutes.
FAQ
Q: What do I do if the error Unsupported Kafka table definition is reported during a major engine version upgrade?
A: Kafka tables in the current version do not support using the DEFAULT keyword to define default field values. This causes the kernel package to fail to start. To resolve this issue, perform the following steps:
Run the
select create_table_query from system.tables where engine = 'Kafka'statement to find all Kafka tables.Back up the Data Definition Language (DDL) statements of the tables.
Delete the tables.
Re-create the tables.
ImportantWhen you re-create the tables, do not use the
DEFAULTkeyword to define default field values.
Q: What do I do if the error Unsupported MaterializedMySQL table definition is reported during a major engine version upgrade?
A: The configuration parameters of the MaterializedMySQL engine in the current version are incompatible with those in the source cluster's version. To resolve this issue, perform the following steps:
Run the
select name from system.databases where engine = 'MaterializedMySQL'statement to find the databases that use the MaterializedMySQL engine.Back up the DDL statements of the databases.
Delete the databases.
Upgrade the kernel version.
Adjust the backed-up DDL statements to be compatible with the target version, and then re-create the databases that use the MaterializedMySQL engine.
Q: What do I do if the error Unsupported table definition other than 20.3: Nullable(Array(*))/SecondaryIndex(KEY definition exists) is reported during a major engine version upgrade?
A: If your cluster version is 20.3, you might be using some features developed by Alibaba Cloud, such as:
Defining a table field of the Nullable(Array(*)) type.
A secondary index defined using the KEY keyword.
Because these features have not been merged back into open source ClickHouse, instance versions later than 20.8 do not include them. Adjust the relevant tables before you upgrade. You can perform the following operations:
Validate the compatibility between the source cluster version and the target version.
ImportantBecause the version span between 20.3 and the current version is large, you must perform a thorough validation before you implement the upgrade to avoid affecting your services.
Delete the field modified with
Nullable(Array(*))and then add the field again.Delete the secondary index that is defined with the KEY keyword. After the upgrade is complete, add a skipping index to the table again.
ImportantThe two types of indexes have different implementation principles, which may cause performance differences.
References
Data migration and synchronization.
Upgrading a major engine version through migration