Flink is one of the most active stream computing engines, and the version upgrade frequency of Flink is relatively high. You can upgrade the engine version of your deployment to a new version to use updated or new features. This topic describes how to upgrade the engine version of a Realtime Compute for Apache Flink deployment.
Precautions
Before a new engine version of Realtime Compute for Apache Flink is released, various compatibility tests are performed. In principle, minor versions of the same major version are compatible with each other. However, the upgrade compatibility between major versions is not ensured. For more information about engine versions, see Engine version and meaning of each digit in a version number.
When you change the engine version of a deployment, take note of the following points:
If you upgrade the engine version of your deployment to a later minor version of the same major version, such as the upgrade from vvr-4.0.15-flink-1.13 to vvr-4.0.18-flink-1.13, the deployment after the upgrade is compatible with the state data before the upgrade and can use checkpoints or savepoints that are generated before the upgrade.
If you upgrade the engine version of your deployment to a later major version, such as the upgrade from vvr-4.0.15-flink-1.13 to vvr-6.0.2-flink-1.15, the deployment after the upgrade is incompatible with the state data before the upgrade. You must restart your deployment without using the state data.
The versions of the Flink dependencies in an SQL deployment or a DataStream deployment must be the same as the Flink version that is selected for the deployment.
Procedure
Step 1: Back up the deployment state
You can click the name of a deployment on the Deployments page and click the State tab to view the state set of the deployment. For more information, see View the state generation overview.
If your deployment uses stateful computing, you must consider whether the state data can be reused for the deployment after the upgrade.
Before you upgrade the engine version of your deployment, manually create a savepoint for the deployment. This can help you quickly roll back your deployment if an exception occurs during the upgrade. For more information, see Manually create a savepoint.
ImportantIn principle, minor versions of the same major version are compatible with each other. However, the upgrade compatibility between major versions is not ensured. If the versions before and after the upgrade are compatible with each other, the new version of the engine can read the savepoints that are generated by the old version of the engine. The old version of the engine may not be able to read the savepoints that are generated by the new version of the engine.
Realtime Compute for Apache Flink that uses Ververica Runtime (VVR) 6.X or later supports two savepoint formats: native format and standard format. The native format supports faster savepoint generation. The standard format provides better compatibility. Therefore, if you want to upgrade the engine version of a deployment to a later minor version of the same major version, we recommend that you use the native format for savepoint generation. If you want to upgrade the engine version of a deployment to a later major version, we recommend that you use the standard format.
If your deployment is stateless, proceed to the next step to change the engine version of the deployment.
Step 2: Change the engine version of a deployment
When you change the engine version of a deployment, we recommend that you select a stable version or a recommended version. Known defects and related issues in other versions are fixed in the stable versions and recommended versions. These versions provide the latest features and higher stability.
SQL deployment
Go to the entry point for upgrading the engine version of the desired deployment.
On the right side of the draft edit section, click the Configurations tab. In the Configurations panel, select a desired version from the Engine Version drop-down list, and then click Deploy in the upper-right corner of the SQL Editor page.
On the Deployments page, click the name of the desired deployment. In the Basic section of the Configuration tab, confirm that the engine version of the deployment is changed to the engine version that you selected.
On the Deployments page, find the deployment whose engine version is changed and click Start in the Actions column to make the configuration of the deployment take effect.
DataStream deployment
Go to the entry point for upgrading the engine version of the desired deployment.
On the Configuration tab, click Edit on the right side of the Basic section.
Select the desired version from the Engine Version drop-down list, and click Save in the upper-right corner of the Configuration tab.
On the Deployments page, find the deployment whose engine version is changed and click Start in the Actions column to make the configuration of the deployment take effect.
Step 3: Verify the deployment data
In most cases, if the deployment starts as expected and the first checkpoint is generated after the upgrade, the upgrade is considered initially successful. We recommended that you observe the correctness of business data to ensure the integrity of the upgrade results.
If the deployment runs as expected and the data correctness is verified, you can manually delete the savepoints that are generated before the upgrade. For more information, see Manually delete a specified savepoint.
Rollback for upgrade failures
If the deployment cannot be started or the business data is abnormal after the upgrade, we recommend that you immediately cancel the deployment, change the engine version of the deployment to the version before the upgrade, and then use the savepoint that is saved before the upgrade to restore the business. You can also submit a ticket to provide feedback.
If you cannot track the entire upgrade process, we recommend that you configure a deployment failure alert to notify you of exceptions at the earliest opportunity. For more information, see Configure alert rules.