Create application-consistent snapshots to ensure application consistency - Elastic Compute Service

On a running Elastic Compute Service (ECS) instance, some data of applications, such as databases, may still be stored in the memory or being written to storage devices. Snapshots cannot record the data that is not written to disks, which may cause application data loss or data inconsistency when you want to restore data by using snapshots. Alibaba Cloud combines the snapshot service with Cloud Assistant to provide the application-consistent snapshot feature, which ensures data consistency within your applications when you restore data from snapshots.

Background information

If you enable the application-consistent snapshot feature when you create a snapshot-consistent group, the system creates application- or file system-consistent snapshots based on the business scenario. The following table describes the two types of snapshots.

Item	Application-consistent snapshot	File system-consistent snapshot
Data consistency	Ensure data consistency within stored data and applications, such as databases.	Ensure data consistency within file systems.
Implementation	In Linux, the application-consistent snapshot feature is implemented by using custom shell scripts based on your applications. When you use custom scripts, Alibaba Cloud does not guarantee application consistency. In Windows, built-in features, such as Volume Shadow Copy Service (VSS), are provided to implement the application-consistent snapshot feature. Note VSS is a Microsoft Windows backup and recovery technology that creates snapshots when files or volumes are in use. For more information, see Volume Shadow Copy Service.	In Linux, if no shell script is configured or a shell script fails to be run, file system-consistent snapshots are created. In Windows, if the Contain Writers by Default option is deselected, file system-consistent snapshots are created. Note Writers are a key component in the VSS architecture. The Windows operating system provides Writers to ensure data consistency during backup. For more information, see Volume Shadow Copy Service.
Supported scenarios	Application-consistent snapshots are suitable for applications that require data consistency, such as databases and critical business systems.	File system-consistent snapshots are suitable for scenarios in which overall file system consistency is required, such as file servers and document management systems.
Tags	APPConsistent:True	FsConsistent:True

How the application-consistent snapshots work

The following section describes how the application-consistent snapshots work.

Linux instance

After the application-consistent snapshot feature is enabled for a Linux instance, the system is in the file system-consistent state by default. After you upload the shell scripts, including prescript.sh and postscript.sh scripts, that are prepared in advance, the system checks the application consistency based on the scripts. If the check is passed, snapshots are marked as application-consistent snapshots. If the check fails, snapshots are marked as file system-consistent snapshots.

The following table describes the prescript.sh and postscript.sh scripts.

Script name	Functionality	Type	Path	Permissions	Content
prescript.sh	Used to suspend all write operations to an application before you perform operations that may affect the application. This prevents data inconsistency due to new data generated during the operations.	Shell	/tmp/prescript.sh	Only the root user has read, write, and execute permissions on the scripts.	You must compile the custom scripts based on your business requirements. For example, compile the scripts by following the procedure in Step 1: Create the prescript.sh and postscript.sh scripts.
postscript.sh	After an operation that may affect an application, such as data backup, is complete, the postscript.sh script is run to restore the application and resume write operations.	Shell	/tmp/postscript.sh

Important

The script name, script type, and storage path for creating application-consistent snapshots are fixed. Use the fixed values when you compile the custom scripts. Otherwise, the scripts may fail to run.

Enable the application-consistent snapshot feature for the ECS instance for which you want to create snapshots and check whether Cloud Assistant is installed on the instance.
Note
If you do not install Cloud Assistant on the instance and you select the Enable Application-consistent Snapshot option, the required Cloud Assistant plug-in is automatically installed on the instance.
Cloud Assistant accesses the ECS instance to automatically run relevant commands, run the prescript.sh script, suspend I/O operations, and create snapshots.
- The script settings are correct and the script runs as expected to create application-consistent snapshots.
- If the script settings, such as the permissions, storage path, or script name, are invalid or the script fails to be run, file system-consistent snapshots are created instead of application-consistent snapshots.
Cloud Assitant accesses the postscript.sh script to resume I/O operations.

Windows instance

After you enable the application-consistent snapshot feature for a Windows instance, Cloud Assistant performs an application-consistent check together with VSS. If the check is passed, snapshots are marked as application-consistent snapshots. If the check fails, snapshots are marked as file system-consistent snapshots.

Enable the application-consistent snapshot feature for the ECS instance for which you want to create snapshots and check whether Cloud Assistant is installed on the instance.
Note
If you do not install Cloud Assistant on the instance and you select the Enable Application-consistent Snapshot option, the required Cloud Assistant plug-in is automatically installed on the instance.
Cloud Assistant accesses the ECS instance to automatically run relevant commands, suspend I/O operations, and create snapshots.
1. Cloud Assistant invokes VSS and suspends all I/O operations.
2. Check whether the Contain Writers by Default option is selected.
  - If the option is selected, application-consistent snapshots are created.
  - If the option is deselected, file system-consistent snapshots are created.
Resume I/O operations.

Limits

Application-consistent snapshots are supported only by Enterprise SSDs (ESSDs) for which the multi-attach feature is disabled.
You can create application-consistent snapshots for cloud disks that are attached to the same ECS instance, but not to different ECS instances.

Prerequisites

An ECS instance is created and runs one of the following operating system versions:
- Windows: Windows Server 2012 R2 or later.
- Linux: CentOS 7.6 or later, Ubuntu 18.04 or later, or Alibaba Cloud Linux 2.
The ECS instance is in the Running state. Cloud Assistant is in the Normal state on the instance. For information about how to view the status of Cloud Assistant, see View the status of Cloud Assistant and handle anomalies.
A Resource Access Management (RAM) role is attached to the ECS instance and a custom policy that contains the permissions on application-consistent snapshots is attached to the RAM role. For more information, see Create an instance RAM role and attach the RAM role to an ECS instance.
Note
When you create application-consistent snapshots, you must invoke Cloud Assistant to access the ECS instance and run commands. Before you invoke Cloud Assistant, grant Cloud Assistant the required permissions by using the RAM role.
- Configure a custom RAM role, such as AppSnapshotRoleName.
- Attach a custom policy to the RAM role. The following sample code shows a custom policy, which indicates that the RAM role has the permissions to query snapshots, create snapshots, add tags, and query disk information.
```
{
    "Version": "1",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecs:DescribeSnapshot*",
                "ecs:CreateSnapshot*",
                "ecs:TagResources",
                "ecs:DescribeDisks"
            ],
            "Resource": [
                "*"
            ],
            "Condition": {}
        }
    ]
}
```

Procedure

Use the ECS console

Linux instance

Upload the prepared prescript.sh and postscript.sh scripts to the ECS instance in advance.
For more information, see Use Workbench to upload files to or download files from an ECS instance.
Note
For information about the prescript.sh and postscript.sh scripts, see How the application-consistent snapshots work.
Go to ECS console - Snapshot-consistent Groups.
In the top navigation bar, select the region and resource group of the resource that you want to manage.
On the Snapshot-consistent Groups tab, click Create Snapshot-consistent Group.
In the Create Snapshot dialog box, configure the parameters described in the following table to create a snapshot-consistent group.
1. By default, the Resource Type parameter is set to Instance.
2. Select a Linux instance from the Select Instances drop-down list. Then, in the Select Cloud Disks section, select the ESSDs for which you want to create snapshots from the instance.
3. Expand Advanced Settings and configure application-consistent snapshot settings.
  - If you select Enable Application-consistent Snapshot and correctly configure scripts, application-consistent snapshots are created.
  - If you select Enable Application-consistent Snapshot but do not configure or incorrectly configure scripts, file system-consistent snapshots are created.
    Note
    If you do not install Cloud Assistant on the instance and you select Enable Application-consistent Snapshot, the required Cloud Assistant plug-in is automatically installed on the instance.
  - You can also select Enable to enable the File System I/O Suspension and Resume feature and specify a timeout period.
    Note
    If you enable the File System I/O Suspension and Resume feature, the system suspends I/O operations to file systems when snapshots are being created to prevent data inconsistency caused by ongoing I/O operations. If you set the Timeout Period parameter to a value greater than 16 and the I/O operation suspension is not complete within the period, the I/O operation suspension fails, which may cause data inconsistency issues.
  - For information about the parameters, see the Procedure section of this topic.
4. Click OK.
  After the snapshot-consistent group is created, a message that contains the Cloud Assistant command ID and the task ID is displayed. You can check whether application-consistent snapshots are created based on the task ID.
On the Command Execution Result tab of the ECS Cloud Assistant page, click the task ID and check whether application-consistent snapshots are created.
The value of ExitCode is 0 as shown in the preceding figure. This indicates that application-consistent snapshots are created as expected on Cloud Assistant. In this case, the ID of the snapshot-consistent group appears in the command output.
Note
If the value of ExitCode is not 0, an error occurred. Troubleshoot the issue based on the value in the ExitCode column. For more information, see the Error codes section of this topic.
View information about the created snapshot-consistent group and snapshots in the group.
1. Go to ECS console - Snapshot-consistent Groups.
2. On the Snapshot-consistent Groups tab, find the snapshot-consistent group that you created and click the ID of the snapshot-consistent group to view snapshot details.
3. In the Snapshot Information section, check whether a snapshot is an application-consistent snapshot or a file system-consistent snapshot based on the tag.
  - If the AppConsistent:True tag appears, an application-consistent snapshot is created.
  - If the FsConsistent:True tag appears, a file system-consistent snapshot is created.

Windows instance

Go to ECS console - Snapshot-consistent Groups.
In the top navigation bar, select the region and resource group of the resource that you want to manage.
On the Snapshot-consistent Groups tab, click Create Snapshot-consistent Group.
In the Create Snapshot dialog box, configure the parameters described in the following table to create a snapshot-consistent group.
1. By default, the Resource Type parameter is set to Instance.
2. Select a Windows instance from the Select Instances drop-down list. Then, in the Select Cloud Disks section, select the ESSDs for which you want to create snapshots from the instance.
3. Expand Advanced Settings and configure application-consistent snapshot settings.
  - If you select Enable Application-consistent Snapshot and Contain Writers by Default, application-consistent snapshots are created.
  - If you select only Enable Application-consistent Snapshot, file system-consistent snapshots are created.
    Note
    If you do not install Cloud Assistant on the instance and you select Enable Application-consistent Snapshot, the required Cloud Assistant plug-in is automatically installed on the instance.
  - For information about the parameters, see the Procedure section of this topic.
4. Click OK.
  After the snapshot-consistent group is created, a message that contains the Cloud Assistant command ID and the task ID is displayed.
On the Command Execution Result tab of the ECS Cloud Assistant page, click the task ID and check whether application-consistent snapshots are created.
The value of ExitCode is 0 as shown in the preceding figure. This indicates that application-consistent snapshots are created as expected on Cloud Assistant. In this case, the ID of the snapshot-consistent group appears in the command output.
Note
If the value of ExitCode is not 0, an error occurred. Troubleshoot the issue based on the value in the ExitCode column. For more information, see the Error codes section of this topic.
View information about the created snapshot-consistent group and snapshots in the group.
1. Go to ECS console - Snapshot-consistent Groups.
2. On the Snapshot-consistent Groups tab, find the snapshot-consistent group that you created and click the ID of the snapshot-consistent group to view snapshot details.
3. In the Snapshot Information section, check whether a snapshot is an application-consistent snapshot or a file system-consistent snapshot based on the tag.
  - If the AppConsistent:True tag appears, an application-consistent snapshot is created.
  - If the FsConsistent:True tag appears, a file system-consistent snapshot is created.

Call an API operation

Call the RunCommand operation to use Cloud Assistant to create application-consistent snapshots for one or more ECS instances.

Configure parameters, such as RegionId, Type, CommandContent, and InstanceId, based on your business requirements. The following section provides CommandContent samples and describes parameters specified in the command content.

Linux instance

acs-plugin-manager --exec --plugin app-snapshot-plugin --params=-RamRoleName=\"AppSnapshotRoleName\",-EnableFsFreeze=true,-TimeoutInSeconds=30,-PreScriptPath=\"/tmp/prescript.sh\",-PostScriptPath=\"/tmp/postscript.sh\",-ExcludeDiskId=\"\",-Name=\"LinuxApp1\",-Description=\"LinuxApp\"

acs-plugin-manager --exec --plugin app-snapshot-plugin indicates the app-snapshot-plugin plug-in used to run Cloud Assistant commands.

--params= indicates the configuration parameters of the plug-in. The following table describes these parameters.

Parameter	Type	Required	Description
RamRoleName	String	Yes	The RAM role attached to the ECS instance.
EnableFsFreeze	Boolean	No	Specifies whether to enable the Linux FsFreeze feature to set the file systems to the read-only state before snapshots are created. Default value: True.
TimeoutInSeconds	Integer	No	The timeout period for I/O operations. Unit: seconds. Default value: 30.
PreScriptPath	String	No	The path in which the prescript.sh script is stored. Example: /tmp/prescript.sh. The prescript.sh script must meet the following requirements: Permissions: Set the permission value to 700, which indicates that only the root user has the read, write, and execute permissions. Script content: Compile the custom script content based on your business requirements. Important If you create application-consistent snapshots for Linux instances, this parameter is required. If the script contains invalid settings, such as permissions, paths, or file names, file system-consistent snapshots are created instead of application-consistent snapshots.
PostScriptPath	String	No	The path in which the postscript.sh script is stored. Example: /tmp/postscript.sh. The postscript.sh script must meet the following requirements: Permissions: Set the permission value to 700, which indicates that only the root user has the read, write, and execute permissions. Script content: Compile the custom script content based on your business requirements. Important If you create application-consistent snapshots for Linux instances, this parameter is required. If the script contains invalid settings, such as permissions, paths, or file names, file system-consistent snapshots are created instead of application-consistent snapshots.
ExcludeDiskId	String	No	The cloud disks to exclude from the snapshots.
Name	String	Yes	The name of the snapshot-consistent group.
Description	String	No	The description of the snapshot-consistent group.

Windows instance

acs-plugin-manager --exec --plugin app-snapshot-plugin-win --params=-RamRoleName=\"AppSnapshotRoleName\",-EnableWriters=true,-ExcludeDiskId=\"\",-Name=\"APPSnapshot-1\",-Description=\"AppSnapshot\"

acs-plugin-manager --exec --plugin app-snapshot-plugin-win indicates the app-snapshot-plugin-win plug-in used to run Cloud Assistant commands.

--params= indicates the configuration parameters of the plug-in. The following table describes these parameters.

Parameter	Type	Required	Description
RamRoleName	String	Yes	The RAM role attached to the ECS instance.
EnableWriters	Boolen	No	Specifies whether to create application-consistent snapshots. Valid values: true false Default value: true.
ExcludeDiskId	String	No	The cloud disks to exclude from the snapshots.
Name	String	Yes	The name of the snapshot-consistent group.
Description	String	No	The description of the snapshot-consistent group.

Call the DescribeInvocationResults operation based on the return value of InvokeId.

Check whether application-consistent snapshots are created.

ExitCode returns an error code.
- A value of 0 indicates that application-consistent snapshots are created.
- If the value of ExitCode is not 0, an error occurred. Fix the error based on the returned error code in the ExitCode column. For more information, see the Error codes section of this topic.

Output is the output after a command is run. If the snapshot-consistent group is created, the group information, such as the ID of the group, is displayed.

[time=\"2025-03-04 16:09:25.8200239\"][message=\"Finish SnapshotGroup=ssg-2zefohc25d7n1grq**** Creation, TotalCost=3.8204978s, QueryCost=2.9307022s\"]\n[level=\"info\"][time=\"2025-03-04 16:09:25.8200239\"][message=\"Prepare to Thaw FileSystem or Applications\"]\n[requestor=\"\n\"]\n[level=\"info\"][time=\"2025-03-04 16:09:27.7133096\"][message=\"Thaw Write Request Done\"]\n[level=\"info\"][time=\"2025-03-04 16:09:28.0355042\"][message=\"Tag snapshots with AppConistent\"]\n[level=\"info\"][time=\"2025-03-04 16:09:28.0365941\"][message=\"Take AppConsistent snapshots successfully\"]

Error codes

Error code (ExitCode)	Description
0	The snapshot is created as expected.
1	One or more conditions are not met. One of the following errors may occur: The cloud disk category is not supported. The snapshot name is invalid. Network connectivity issues occurred. No RAM role is attached to the instance. The operating system of the instance is not supported.
2	The type or number of the parameters that follow the `--params` string is invalid.
3	One of the following errors may occur: No ESSDs are attached to the instance. The RAM role attached to the instance does not have permissions to call snapshot-related API operations.
4	The snapshot-consistent group cannot be created.
5	The snapshot-consistent group is not in the expected state.
6	The request to create the snapshot-consistent group timed out.
7	A disk snapshot in the snapshot-consistent group is not in the expected state.
8	Tags cannot be added to the snapshot.
9	The prescript.sh script cannot be run.
10	The postscript.sh script cannot be run.
11	I/O operations cannot be suspended.
12	I/O operations cannot be resumed.
13	No RAM role is attached to the instance.
14	The number of snapshots exceeded the upper limit.
15	The snapshot is not in the expected state.
16	The previous snapshot is being created, and new snapshots cannot be created.
255	An unknown error occurred.

References

You can create application-consistent snapshots for MySQL or SQL Server databases. For more information, see Best practices for creating application-consistent snapshots for a Linux instance on which MySQL is deployed or Best practices for creating application-consistent snapshots for a Windows instance on which SQL Server is deployed.