Use ApplyDataFlowAutoRefresh to add an AutoRefresh configuration to a dataflow - Cloud Parallel File Storage - Alibaba Cloud - Cloud Parallel File Storage

Configures auto-refresh for a specified data flow.

Operation description

This operation applies only to CPFS file systems.
Only CPFS 2.2.0 and later support data flows. You can view the version information on the file system details page in the console.
You can add auto-refresh configurations only to data flows that are in the Running state.
You can add up to five auto-refresh configurations for a single data flow.
Creating an auto-refresh configuration typically takes 2 to 5 minutes. You can call the DescribeDataFlows operation to query the data flow status.
Auto-refresh relies on EventBridge to collect object modification events from the source OSS storage. You must first activate EventBridge.

Note
The event buses and event rules created by CPFS in EventBridge contain the Create for cpfs auto refresh description. Do not modify or delete these event buses and event rules. Otherwise, auto-refresh cannot work properly.
Auto-refresh applies to prefixes specified by the RefreshPath parameter. When you configure auto-refresh for a prefix in a CPFS data stream, an event bus is created on the user side, and an event rule is created for the prefix of the source OSS bucket. When an object in the prefix of the source OSS bucket is modified, an OSS event is generated in EventBridge and processed by the CPFS data stream.
After auto-refresh (AutoRefresh) is configured, if data in the source storage changes, the changed metadata undergoes automatic synchronization to the CPFS file system. The changed data is loaded on demand when you access the files, or you can start a data stream node to load the data.
The auto-refresh interval (AutoRefreshInterval) specifies the time interval at which CPFS checks whether data in the prefix of the source OSS bucket is updated. If data is updated, an auto-refresh node is started. If the frequency of object modification events in the source OSS bucket exceeds the processing capability of the CPFS data stream, auto-refresh nodes accumulate, metadata updates are delayed, and the data stream enters the Misconfigured state. To resolve this issue, you can upgrade the data stream specifications or reduce the OSS modification frequency.

Try it now

Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

Test

RAM authorization

The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.
API: The API that you can call to perform the action.
Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.
Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.
- For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.
- For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.
Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.
Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

Action

Access level

Resource type

Condition key

Dependent action

nas:ApplyDataFlowAutoRefresh

update

*DataFlow

acs:nas:{#regionId}:{#accountId}:filesystem/{#fileSystemId}

None

Request parameters

Parameter	Type	Required	Description	Example
FileSystemId	string	Yes	The ID of the file system.	cpfs-099394bd928c****
DataFlowId	string	Yes	The ID of the data flow.	df-194433a5be31****
AutoRefreshs	array<object>	Yes	The collection of auto-refresh configurations.
	object	No
RefreshPath	string	Yes	The auto-refresh directory. CPFS automatically checks whether the source data in this directory is updated and automatically imports the updated data. Limits: The length is 2 to 1,024 characters. It must be encoded in UTF-8. It must start and end with a forward slash (/). Note The directory must already exist in CPFS and must be located in a fileset for which the data flow feature is enabled.	/prefix1/prefix2/
AutoRefreshPolicy	string	No	The auto-refresh policy. This policy determines how data updates from the source are imported to CPFS after the source data is updated. Valid values: None: Data updates from the source are not automatically imported to CPFS. You can import data updates from the source by using data flow tasks. ImportChanged: Data updates from the source are automatically imported to CPFS. Valid values: ImportChanged : Data updates from the source are automatically imported to CPFS. None : Data updates from the source are not automatically imported to CPFS.	None
AutoRefreshInterval	integer	No	The auto-refresh interval. CPFS checks whether data in the directory is updated at this interval. If data is updated, an auto-refresh task is started. Unit: minutes. Valid values: 10 to 525600. Default value: 10.	10
DryRun	boolean	No	Specifies whether to perform a dry run on the creation request. A dry run checks whether the parameters are valid and whether resources are available, without actually creating the instance or incurring fees. Valid values: true: performs a dry run without creating the instance. The system checks whether required parameters are specified, the request format is valid, business limits are met, and NAS resources are available. If the dry run fails, an error is returned. If the dry run succeeds, the HTTP status code 200 is returned, but the FileSystemId is empty. false: sends a regular request. If the check passes, the instance is created. Default value: false.	false
ClientToken	string	No	The client token that is used to ensure the idempotence of the request. You can generate the token on your client and ensure that the token is unique across different requests. The client token can contain only ASCII characters and cannot exceed 64 characters in length. For more information, see How to ensure idempotence. Note If you do not specify this parameter, the system automatically uses the RequestId of the API request as the client token. The RequestId of each API request may be different.	123e4567-e89b-12d3-a456-42665544****

Response elements

Element	Type	Description	Example
	object
RequestId	string	The ID of the request.	98696EF0-1607-4E9D-B01D-F20930B6****

Examples

Success response

JSON format

{
  "RequestId": "98696EF0-1607-4E9D-B01D-F20930B6****"
}

Error codes

HTTP status code	Error code	Error message	Description
400	IllegalCharacters	The parameter contains illegal characters.	The parameter contains illegal characters.
400	MissingFileSystemId	FileSystemId is mandatory for this action.	The FileSystemId parameter is not specified.
400	MissingDataFlowId	DataFlowId is mandatory for this action.	The DataFlowId parameter is not specified.
403	OperationDenied.InvalidState	The operation is not permitted when the status is processing.	The operation is not permitted when the status is processing.
403	InvalidFileSystem.AlreadyExisted	The specified file system already exists.	The specified file system already exists.
403	OperationDenied.DependencyViolation	The operation is denied due to dependancy violation.	The conditions for performing this operation are not met.
403	OperationDenied.NestedDir	The operation is denied due to nested directory.	Failed to create the automatic update configuration because nested directories exist.
403	OperationDenied.ConflictOperation	The operation is denied due to a conflict with an ongoing operation.	The data flow cannot be created due to conflicts with current file system operations.
403	OperationDenied.DataFlowNotSupported	The operation is not supported.	The file system does not support data flow.
404	InvalidFileSystem.NotFound	The specified file system does not exist.	The specified file system does not exist.
404	InvalidDataFlow.NotFound	The specified data flow does not exist.	The specified data flow does not exist.
404	InvalidRefreshPath.InvalidParameter	Refresh path is invalid.	The path of the specified auto-refresh directory is invalid.
404	InvalidRefreshPath.NotFound	Refresh path does not exist.	The specified auto-refresh directory does not exist.
404	InvalidRefreshPolicy.InvalidParameter	Refresh policy is invalid.	The auto-refresh policy is invalid.
404	InvalidRefreshInterval.OutOfBounds	Refresh interval is out of bounds.	The specified RefreshInterval is not within the valid range.
404	InvalidRefreshPath.AlreadyExist	The refresh path already exists.	The specified auto-refresh directory already exists.
404	InvalidRefreshPath.TooManyPaths	The number of refresh paths exceeds the limit.	The number of queried auto-refresh directories exceeds the limit.

See Error Codes for a complete list.

Release notes

See Release Notes for a complete list.