Realtime Compute for Apache Flink:Upgrade the engine version of a deployment

Version compatibility

In principle, minor versions within the same major version are compatible with each other. Checkpoints and savepoints generated before the upgrade remain usable.

For details on version numbering, see the "Engine version and meaning of each digit in a version number" section of Engine version. For the upstream compatibility matrix, see the "Compatibility Table" in Upgrading Applications and Flink Versions.

Savepoint format selection (Ververica Runtime 6.X and later)

Realtime Compute for Apache Flink that uses Ververica Runtime (VVR) 6.X or later supports two savepoint formats:

Dependency version alignment

The versions of Realtime Compute for Apache Flink dependencies in an SQL deployment or a DataStream deployment must be the same as the Realtime Compute for Apache Flink version that is selected for the deployment.

BlinkPlanner compatibility (Apache Flink 1.13.0 and later)

Apache Flink 1.13.0 and later use BlinkPlanner as the default SQL Planner. BlinkPlanner was contributed to the Apache Flink community by Alibaba Group. If you are migrating from an Apache Flink version earlier than 1.13.0 to VVR 4.0 or later (based on Apache Flink 1.13.0), some syntaxes and APIs may be incompatible. For details, see Apache Flink 1.13.0 Release Announcement.

Step 1: Clone the draft and set the target engine version

Select a stable or recommended engine version. These versions have known defects fixed and provide the latest features with higher stability.

Step 2: Back up the deployment state

In the left-side navigation pane, choose O&M > Deployments. Click the deployment name and check the State tab to view the state set. For details, see the "View the state generation overview" section in Manage a state set.

• Stateful deployments: Create a savepoint manually before upgrading. This savepoint serves as your rollback point if the upgrade fails. For instructions, see the "Manually create a savepoint" section of Manage a state set.

  Important

  • In principle, minor versions of the same major version are compatible with each other. However, cross-major-version compatibility is not guaranteed. If the versions are compatible, the new engine can read savepoints from the old engine. The old engine may not be able to read savepoints from the new engine.

  • For VVR 6.X or later, use the native format for minor version upgrades and the standard format for major version upgrades.

• Stateless deployments: Skip to the next step.

Step 3 (optional): Cancel the original deployment

In the left-side navigation pane, choose O&M > Deployments. Find the original deployment and click Cancel in the Actions column. For details, see Cancel a deployment.

If your downstream operators support idempotent writes, or if duplicate data is acceptable for your business, running both the original and the new deployment simultaneously during the transition is also an option.

Step 4: Start the new deployment

• Stateful deployments: In the Start Job panel, select Resume Mode, then select Specific State, and choose the savepoint created in Step 2.

• Stateless deployments: In the Start Job panel, select Initial Mode.

For full instructions, see Start a deployment.

Step 5: Verify the upgrade and clean up

1. Confirm the first checkpoint: After the new deployment starts, wait for the first checkpoint to complete. A successful first checkpoint indicates the upgrade is initially successful.

2. Monitor business data: Observe the correctness of business data over a period of time to confirm the upgrade is fully successful.

3. Clean up: After you verify that the deployment runs correctly and the data is accurate, delete the original deployment and the pre-upgrade savepoint. For instructions on deleting a savepoint, see the "Manually delete a specified savepoint" section in Manage a state set.