Checkpoints and savepoints - Realtime Compute for Apache Flink

System checkpoints and savepoints are collectively known as state sets. This topic describes how to manage the state sets for your jobs, including how to create, view, and delete them, and how to restore a job from a specific state.

Background information

Concept

Details

System checkpoint

(Corresponds to Checkpoint in Apache Flink)

The lifecycle is fully managed by the Flink system. You can configure the checkpointing interval, Checkpointing Timeout Time, and min interval between checkpoints, and view their generation status. You cannot manually create or delete system checkpoints. By default, a job retains one system checkpoint as part of the state set for job recovery. For more information about parameter configuration, see Configure job deployment information.

Savepoint

(Corresponds to Savepoint in Apache Flink)

The lifecycle is fully managed by the user in the following ways:

You can trigger a savepoint while a job is running or stopping. You can also trigger a savepoint once or on a schedule on the Realtime Compute for Apache Flink platform.
You can delete a savepoint manually. You can also configure a one-time or scheduled cleanup on the Realtime Compute for Apache Flink platform.
When you start a job, you can restore its state from a specific savepoint.

Note

Even with scheduled generation or cleanup, you manage the lifecycle by defining the rules the system follows.

The following table describes the operations you can perform on system checkpoints and savepoints, along with their use cases and links to relevant documentation.

Actions	Use case	Documentation
Manually create a savepoint	You may need to manually create a savepoint at a specific time (while the job is running or stopping) and restore the job from it later. Important Manually created savepoints can only be deleted manually and are not affected by automatic cleanup rules. Selecting Stop With Savepoint when stopping a job reduces the amount of data to reprocess, but increases the job's stop time.	Manually create a savepoint Stop a job
Configure scheduled automatic creation of savepoints	To automatically create savepoints on a schedule, configure a creation interval. The system then creates savepoints for the running job based on this rule, eliminating the need for manual creation.	Configure scheduled automatic creation of savepoints
View the state generation overview	Check the overview to view the generation history for system checkpoints and savepoints. This information helps you quickly assess the job's state persistence status.	View the state generation overview
Restore a job from a specified savepoint (including from another job)	To restore a job from a specific savepoint, select it when you start the job. Note You can share savepoints between jobs, but you must ensure their states are compatible. This is useful for A/B testing.	Restore a job from a specified savepoint (including from another job)
Configure automatic cleanup rules for savepoints	To automatically clean up savepoints instead of deleting them manually, configure rules based on the number of savepoints to retain or their maximum retention period. After you save the rule, the system automatically cleans up savepoints for the running job based on this rule. Note Automatic cleanup does not apply to manually created savepoints. Manually created savepoints must be deleted manually. If a job has been stopped for a long time, the system will clean up savepoints based on the previously set rules when the job restarts.	Configure automatic cleanup rules for savepoints
Manually delete a specific savepoint	Manually created savepoints are not automatically deleted when a job stops. You must periodically delete outdated savepoints to avoid unnecessary Object Storage Service (OSS) usage and associated costs.	Manually delete a specific savepoint
Configure state-related parameters	Configure parameters such as the state storage system for streams, the state TTL for SQL jobs, and whether to store temporary data generated during savepoint creation on local disks.	Configure state-related parameters

Manually create a savepoint

Note

You can manually create a savepoint only for a job that is in the RUNNING state.

Go to the savepoint creation page for the target job.
1. Log on to the Realtime Compute for Apache Flink console.
2. In the Actions column of the target workspace, click Console.
3. On the O&M > Deployments page, click the name of the target job.
In the upper-right corner of the page, click Create Savepoint.
Select a savepoint format and enter a description.

Savepoint formats include native format and canonical format. The native format is faster to generate and consumes less storage, but it is not compatible across Flink versions. The canonical format takes longer to generate and consumes more storage, but it supports job migration across different Flink versions.
Click Create.

Configure scheduled automatic creation of savepoints

Go to the State page.
1. Log on to the Realtime Compute for Apache Flink console.
2. In the Actions column of the target workspace, click Console.
3. On the O&M > Deployments page, click the name of the target job.
4. Click State.

On the Configuration tab, turn on the Savepoint automatically generated at regular intervals switch and set the schedule.

You can click Quick Setting to select a predefined time interval, or manually enter a cron expression. To enter an expression manually, click each field to modify its value. The following table describes the valid values and supported special characters for the five fields of a cron expression.

Field	Required	Value range	Special characters
Minute	Yes	[0, 59]	* , - /
Hour	Yes	[0, 23]	* , - /
Day of month	Yes	[1, 31]	* , - /
Month	Yes	[1, 12] or [JAN, DEC]	* , - /
Day of week	Yes	[1, 7] or [MON, SUN]. Note If you use the numeric format [1, 7], 1 represents Monday and 7 represents Sunday.	* , - /

Note

The minimum interval is 10 minutes. If you specify an interval shorter than 10 minutes, the system defaults to a 10-minute interval.

Each field supports a set of special characters with specific meanings, as described in the following table.

Special character	Description	Example
`*`	All possible values.	In the Month field, `` means every month. In the Day of week field, `` means every day of the week.
`,`	A list of enumerated values.	In the Minute field, `5,20` triggers the task at 5 and 20 minutes past the hour.
`-`	A range of values.	In the Minute field, `5-20` triggers the task every minute from minute 5 to minute 20.
`/`	An increment for a numeric value.	In the Minute field, `0/15` triggers the task every 15 minutes, starting at 0 minutes past the hour. `3/20` triggers the task every 20 minutes, starting at 3 minutes past the hour.

The following table provides examples of cron expressions.

Example	Description
`15 10 * * *`	Runs at 10:15 AM daily.
`0 12 * * *`	Runs at 12:00 PM (noon) daily.
`0 10,14,16 * * *`	Runs at 10:00 AM, 2:00 PM, and 4:00 PM daily.
`0/30 9-17 * * *`	Runs every 30 minutes between 9:00 AM and 5:00 PM daily.
`* 14 * * *`	Runs every minute from 2:00 PM to 2:59 PM daily.
`0-5 14 * * *`	Runs every minute from 2:00 PM to 2:05 PM daily.
`0/5 14 * * *`	Runs every 5 minutes from 2:00 PM to 2:55 PM daily.
`0/5 14,18 * * *`	Runs every 5 minutes during the 2:00 PM hour (2:00 PM to 2:55 PM) and the 6:00 PM hour (6:00 PM to 6:55 PM) daily.
`0 12 * * WED`	Runs at 12:00 PM (noon) every Wednesday.
`15 10 15 * *`	Runs at 10:15 AM on the 15th of every month.
`10,44 14 * 3 WED`	Runs at 2:10 PM and 2:44 PM on every Wednesday in March.

Important

Field definition: The expression contains five fields: Minute Hour Day-of-month Month Day-of-week.
Union of Day and Week: If both the third field (Day-of-month) and the fifth field (Day-of-week) are specified, they are combined with an OR condition.

For example, 0 12 1 * 5 means the task runs at 12:00 PM on the 1st of the month or on every Friday. It does not mean the task runs only when the 1st of the month is also a Friday.
Question mark (?) is not supported: Do not use the ? character, as the system will report an error or fail to recognize the expression. To ignore a field, you must use *.

View the state overview

Go to the State page.
1. Log on to the Realtime Compute for Flink console.
2. In the Actions column for the target workspace, click Console.
3. On the O&M > Deployments page, click the name of the target deployment.
4. Click State.

On the Overview tab, view the following information.

Category	Information
System checkpoint	Displays information about the last successful and failed system checkpoints, including the ID, completion time, duration, and total storage size.
Savepoint	Displays information about the last successful and failed savepoints, including the savepoint ID, completion time, duration, or failure time.

Restore a job from a savepoint of another job

Note

Only jobs in the CANCELLED state can be restored from a specific savepoint. If your job is RUNNING, stop the job first.

Go to the State page.
1. Log on to the Realtime Compute for Apache Flink console.
2. In the Actions column of the target workspace, click Console.
3. On the O&M > Deployments page, click the name of the target deployment.
4. Click State.
On the History tab, find the target savepoint in the Savepoints list.
In the Actions column of the target savepoint, choose More > Start job from this savepoint.
In the Start Job configuration dialog box, configure the startup settings for the job.

For more information, see Start a job.

Configure automatic cleanup rules for savepoints

Go to the State page.
1. Log on to the Realtime Compute console.
2. In the Actions column of the target workspace, click Console.
3. On the O&M > Deployments page, click the name of the target deployment.
4. Click State.
On the Configuration tab, turn on the Scheduled automatic cleaning of savepoint switch and configure the cleanup rules.

You can configure the Number of historical snapshots saved and the Maximum Retention Period for Historical Savepoints. When a Flink job is running, savepoints are cleaned up if either of these conditions is met.
Click Save Changes.

Delete a savepoint

Go to the State page.
1. Log on to the Realtime Compute console.
2. In the Actions column for the target workspace, click Console.
3. On the O&M > Deployments page, click the name of the target deployment.
4. Click State.
On the History tab, find the target savepoint in the Savepoints list.
In the Actions column of the target savepoint, choose More > Delete This Savepoint.
Click OK.

Set state parameters

Go to the Deployments page of the target deployment.
1. Log on to the Realtime Compute for Apache Flink console.
2. In the Actions column of the target workspace, click Console.
3. On the O&M > Deployments page, click the name of the target deployment.
On the Configuration tab, click Edit on the right side of the Parameters section.
In the Other Configuration section, add the following code and click Save.
```
state.backend.incremental: true
table.exec.state.ttl: 129600000
```
For more information about basic and advanced Flink parameters, see Configure GeminiStateBackend.

Realtime Compute for Apache Flink:Manage state sets

Background information

Manually create a savepoint

Configure scheduled automatic creation of savepoints

View the state overview

Restore a job from a savepoint of another job

Configure automatic cleanup rules for savepoints

Delete a savepoint

Set state parameters

Related documents