All Products
Search

This Product

    Cloud Parallel File Storage:CreateDataFlow

    Document Center

    Cloud Parallel File Storage:CreateDataFlow

    Last Updated:Jul 21, 2026

    Creates a data flow between a CPFS General-purpose or CPFS for Lingjun file system and source storage.

    Operation description

    • This operation applies to the following products:

    ProductFile system ID formatMinimum version that supports data flows
    CPFS General-purposeStarts with cpfs-, such as cpfs-125487****2.2.0 or later
    CPFS for LingjunStarts with bmcpfs-, such as bmcpfs-0015****2.4.0 or later
    Note

    CPFS General-purpose and CPFS for Lingjun share the same set of APIs but differ in parameter values and feature support. Refer to the corresponding section based on your product type.

    • Basic operations

      • You can create a data flow only when the CPFS General-purpose or CPFS for Lingjun file system is in the Running state.

      • A maximum of 10 data flows can be created for a single CPFS General-purpose or CPFS for Lingjun file system.

      • Creating a data flow typically takes 2 to 5 minutes. You can call DescribeDataFlows to check whether the data flow is created.

    • Permissions.

      When you create a data flow, Cloud Parallel File Storage associates the AliyunServiceRoleForNasOssDataflow and AliyunServiceRoleForNasEventNotification service-linked roles. For more information, see CPFS service-linked roles.

    • CPFS General-purpose usage notes.

      This section applies to CPFS General-purpose file systems whose IDs start with cpfs-.

      • Billing

        • Creating a data flow incurs charges based on the data flow bandwidth. For more information, see CPFS General-purpose billing.

        • When you use auto-refresh (AutoRefresh), EventBridge collects object modification events from the source OSS storage, which incurs fees. For more information, see EventBridge billing.

      • Data flow specifications

        • Data flow bandwidth (Throughput) supports three specifications: 600 MB/s, 1200 MB/s, and 1500 MB/s. The data flow bandwidth refers to the maximum transfer bandwidth that the data flow can achieve during data import or export.

        • Creating a data flow consumes one vSwitch IP address used by the Cloud Parallel File Storage General-purpose mount target. Ensure that sufficient vSwitch IP resources are available.

        • Inventory check: When DryRun is set to true, you can verify whether the resources required to create a data flow of the specified specification are available.

      • Fileset

        • The destination of a data flow is a Fileset in the CPFS General-purpose file system. A Fileset is a new directory tree structure in the CPFS General-purpose file system. It is a small file system within the parent file system that has an independent inode space and management capability.

        • The Fileset must already exist when you create a data flow, and it cannot be nested with other Filesets. Only one data flow can be created on a Fileset, corresponding to one source storage.

        • The maximum number of files in a Fileset is 1 million. If the number of files imported from an OSS bucket exceeds this limit, creating new files returns a no space error.

      Note

      If data already exists in the Fileset, the existing data in the Fileset is cleared and replaced with data synchronized from OSS after the data flow is created.

      • Auto-refresh

        • After auto-refresh (AutoRefresh) is configured, when data in the source storage changes, the changed metadata is automatically synchronized to the Cloud Parallel File Storage General-purpose file system. The changed data is loaded on demand when a user accesses the file, or loaded by starting a data flow task to load data.

        • Auto-refresh relies on EventBridge to collect object modification events from the source OSS storage. You must first activate EventBridge.

        • The scope of auto-refresh is a prefix, specified by the RefreshPath parameter. A maximum of 5 auto-refresh directories can be configured for a data flow.

        • The auto-refresh interval (AutoRefreshInterval) specifies the interval at which Cloud Parallel File Storage General-purpose checks whether data updates exist in the specified prefix of the source OSS bucket. If data updates exist, an auto-refresh task is started. When the frequency of object modification events in the source OSS exceeds the processing capacity of the Cloud Parallel File Storage General-purpose data flow, auto-refresh tasks accumulate, metadata updates are delayed, and the data flow status becomes Misconfigured. You can resolve this issue by upgrading the data flow specification or reducing the OSS modification frequency.

        • When auto-refresh is configured for a prefix in the Cloud Parallel File Storage General-purpose data flow, an event bus is created on the user side, and an event rule is created for the prefix of the source OSS bucket. When an object modification occurs in the prefix of the source OSS bucket, an OSS event is generated in EventBridge and processed by the Cloud Parallel File Storage General-purpose data flow.

        Note

        The event bus and event rules created by Cloud Parallel File Storage General-purpose in EventBridge have the description Create for cpfs auto refresh. Do not modify or delete the event bus or event rules. Otherwise, auto-refresh does not work properly.

      • Source storage

        • Only OSS is supported as source storage. The source storage (SourceStorage) of a data flow must be an OSS bucket.

        • Cloud Parallel File Storage General-purpose data flows support both encrypted and non-encrypted access to OSS. When you select encrypted (SSL) access to OSS, confirm that the encryption in transit settings of the OSS bucket support encrypted access.

        • If multiple Cloud Parallel File Storage General-purpose data flows or multiple data flows of the same Cloud Parallel File Storage General-purpose file system use the same OSS bucket as source storage, enable versioning for the OSS bucket to prevent data conflicts when multiple Cloud Parallel File Storage General-purpose file systems export data to the same source.

        • Cross-region OSS data flows are not supported. The OSS bucket must be in the same region as the Cloud Parallel File Storage General-purpose file system.
          Note

          Before creating a data flow, set a tag (key: cpfs-dataflow, value: true) on the source OSS bucket so that the Cloud Parallel File Storage General-purpose data flow can access the data in the bucket. Do not delete or modify this tag during the use of the data flow. Otherwise, the Cloud Parallel File Storage General-purpose data flow cannot access the data in the bucket.

    • CPFS for Lingjun usage notes.

      This section applies to CPFS for Lingjun file systems whose IDs start with bmcpfs-.

      • Source storage

        • Only OSS is supported as source storage. The source storage (SourceStorage) of a data flow must be an OSS bucket.

        • CPFS for Lingjun data flows support both encrypted and non-encrypted access to OSS. When you select encrypted (SSL) access to OSS, ensure that the encryption in transit of the OSS bucket supports encrypted access.

        • If multiple CPFS for Lingjun data flows or multiple data flows of the same CPFS for Lingjun file system use the same OSS bucket as source storage, enable versioning for the OSS bucket to prevent data conflicts when multiple CPFS for Lingjun file systems export data to the same source.

        • Cross-region OSS data flows are not supported. The OSS bucket must be in the same region as the CPFS for Lingjun file system.

        • CPFS for Lingjun 2.6.0 or later supports creating data flows with cross-account OSS.

        • The account ID parameter is required only when you use cross-account OSS.

        • When you use cross-account OSS, authorize the accounts first. For more information, see Cross-account data flow authorization.
          Note

          Before creating a data flow, set a tag (key: cpfs-dataflow, value: true) on the source OSS bucket so that the CPFS for Lingjun data flow can access the data in the bucket. Do not delete or modify this tag during the use of the data flow. Otherwise, the CPFS for Lingjun data flow cannot access the data in the bucket.

      • Data flow restrictions on the file system.

        • In the file system path associated with the data flow, you cannot rename non-empty directories. Otherwise, a Permission Denied or directory not empty error is returned.

        • Use special characters in directory and file names with caution. Supported characters include uppercase and lowercase letters, digits, exclamation marks (!), hyphens (-), underscores (_), periods (.), asterisks (*), and parentheses (()).

        • Excessively long paths are not supported. The maximum path length supported by data flows is 1023 characters.

      • Data flow import restrictions.

        • After Symlink files are imported to CPFS for Lingjun, they are converted to regular files that contain data, and the Symlink information is lost.

        • If the OSS bucket has multiple versions, only the latest version is copied.

        • File names or subdirectory names longer than 255 bytes are not supported.

      • Data flow export restrictions

        • After Symlink files are synchronized to OSS, the files pointed to by the Symlink are not synchronized. Instead, the Symlink becomes a regular empty object with no data.

        • Hardlink files are synchronized to OSS as regular files.

        • Socket, Device, and Pipe files exported to the OSS bucket become regular empty objects with no data.

        • Directory paths longer than 1023 characters are not supported.

    Try it now

    Try this API in OpenAPI Explorer, no manual signing needed. Successful calls auto-generate SDK code matching your parameters. Download it with built-in credential security for local usage.

    Test

    RAM authorization

    The table below describes the authorization required to call this API. You can define it in a Resource Access Management (RAM) policy. The table's columns are detailed below:

    • Action: The actions can be used in the Action element of RAM permission policy statements to grant permissions to perform the operation.

    • API: The API that you can call to perform the action.

    • Access level: The predefined level of access granted for each API. Valid values: create, list, get, update, and delete.

    • Resource type: The type of the resource that supports authorization to perform the action. It indicates if the action supports resource-level permission. The specified resource must be compatible with the action. Otherwise, the policy will be ineffective.

      • For APIs with resource-level permissions, required resource types are marked with an asterisk (*). Specify the corresponding Alibaba Cloud Resource Name (ARN) in the Resource element of the policy.

      • For APIs without resource-level permissions, it is shown as All Resources. Use an asterisk (*) in the Resource element of the policy.

    • Condition key: The condition keys defined by the service. The key allows for granular control, applying to either actions alone or actions associated with specific resources. In addition to service-specific condition keys, Alibaba Cloud provides a set of common condition keys applicable across all RAM-supported services.

    • Dependent action: The dependent actions required to run the action. To complete the action, the RAM user or the RAM role must have the permissions to perform all dependent actions.

    Action

    Access level

    Resource type

    Condition key

    Dependent action

    nas:CreateDataFlow

    create

    *DataFlow

    acs:nas:{#regionId}:{#accountId}:filesystem/{#filesystemId}

    		 None None

    Request parameters

    Parameter

    Type

    Required

    Description

    Example
    FileSystemId

    string

    Yes

    The file system ID.

    • CPFS General-purpose: Must start with cpfs-, such as cpfs-125487****.

    • CPFS for Lingjun: Must start with bmcpfs-, such as bmcpfs-0015****.

    cpfs-099394bd928c****
    ClientToken

    string

    No

    The client token that is used to ensure the idempotence of the request. You can use the client to generate the token, but you must make sure that the token is unique among different requests.

    The token can contain only ASCII characters and cannot exceed 64 characters in length. For more information, see How to ensure idempotence.

    Note

    If you do not specify this parameter, the system automatically uses the value of RequestId as the value of ClientToken. The value of RequestId may be different for each API request.

    123e4567-e89b-12d3-a456-42665544****
    DryRun

    boolean

    No

    Specifies whether to perform a dry run for this request.

    A dry run checks parameter validity and resource availability without actually creating the instance or incurring charges.

    Valid values:

    • true: Sends a dry run request without creating the instance. The check items include whether required parameters are specified, the request format, business limitations, and NAS inventory. If the check fails, the corresponding error is returned. If the check succeeds, HTTP status code 200 is returned, but the DataFlowId is empty.

    • false (default): Sends a normal request. After the check succeeds, the instance is created.

    false
    Description

    string

    No

    The description of the data flow.

    Limits:

    • The description must be 2 to 128 characters in length.

    • The description must start with a letter and cannot start with http:// or https://.

    • The description can contain digits, colons (:), underscores (_), and hyphens (-).

    Bucket01 DataFlow
    SourceSecurityType

    string

    No

    The security protection type of the source storage. If the source storage must be accessed through security protection, specify the security protection type. Valid values:

    • None (default): The source storage does not require security protection for access.

    • SSL: Access is protected by an SSL certificate.

    SSL
    SourceStorage

    string

    Yes

    The access address of the source storage. Format: <storage type>://[<account id>:]<path>.

    Where:

    • storage type: Only oss is supported.

    • account id: Optional. The UID of the account that owns the source storage. This parameter is required when you use cross-account OSS.

    • path: The name of the OSS bucket. Limits:

      • Only lowercase letters, digits, and hyphens (-) are supported. The name must start and end with a lowercase letter or digit.

      • The maximum length is 128 characters.

      • UTF-8 encoding is used.

    Note

    • The OSS bucket must be an existing bucket in the same region.

    • The account id parameter is supported only by CPFS for Lingjun 2.6.0 or later.

    oss://178321033379****:bucket-01
    SourceStoragePath

    string

    No

    The access path within the source storage bucket. Limits:

    • Must start and end with a forward slash (/).

    • Case-sensitive.

    • The length must be 1 to 1023 characters.

    • UTF-8 encoding is used.

    Note

    This parameter is required when the file system type is CPFS for Lingjun.

    /prefix/
    FsetId

    string

    No

    The Fileset ID.

    Note

    This parameter is required when the file system type is CPFS General-purpose.

    fset-1902718ea0ae****
    FileSystemPath

    string

    No

    The directory in the CPFS for Lingjun file system. Limits:

    • Must start and end with a forward slash (/).

    • The directory must be an existing directory on the CPFS for Lingjun file system.

    • The length must be 1 to 1023 characters.

    • UTF-8 encoding is used.

    Note

    This parameter is required when the file system type is CPFS for Lingjun.

    /path/
    AutoRefreshs

    array<object>

    No

    The collection of auto-refresh configurations.

    Note

    This parameter takes effect only when the file system type is CPFS General-purpose.

    object

    No

    RefreshPath

    string

    No

    The auto-refresh directory. CPFS General-purpose registers data modification events from the source storage and checks whether the source data in this directory has been updated. Updated data is automatically imported.

    The default value is empty, which means that data updates in the source storage are not automatically imported to CPFS General-purpose. You must manually create a task to import updates.

    Limits:

    • The length must be 2 to 1024 characters.

    • UTF-8 encoding is used.

    • Must start and end with a forward slash (/).

    • The directory must be an existing directory on the CPFS General-purpose file system and must be located within the Fileset directory of the data flow.

    /prefix1/prefix2/
    AutoRefreshPolicy

    string

    No

    The auto-refresh policy. Specifies the policy for importing data updates from the source storage to CPFS General-purpose after the source data is updated.

    • None (default): Data updates from the source are not automatically imported to CPFS General-purpose. You can import data updates from the source through data flow tasks.

    • ImportChanged: Data updates from the source are automatically imported to CPFS General-purpose.

    Note

    This parameter takes effect only when the file system type is CPFS General-purpose.

    Valid values:

    • ImportChanged :

      Data updates from the source are automatically imported to CPFS General-purpose.

    • None :

      Data updates from the source are not automatically imported to CPFS General-purpose.

    None
    AutoRefreshInterval

    integer

    No

    The auto-refresh interval. CPFS General-purpose checks the directory for data updates at this interval. If data updates exist, an auto-refresh task is started. Unit: minutes.

    Valid values: 10 to 525600. Default value: 10.

    Note

    This parameter takes effect only when the file system type is CPFS General-purpose.

    10
    Throughput

    integer

    No

    The maximum transfer bandwidth of the data flow. Unit: MB/s. Valid values:

    • 600

    • 1200

    • 1500

    Note

    The transfer bandwidth of the data flow must be less than the I/O bandwidth of the file system. This parameter is required when the file system type is CPFS General-purpose.

    Valid values:

    • 600 :

      600MB/s

    • 1200 :

      1200MB/s

    • 1500 :

      1500MB/s

    • 2000 :

      2000MB/s

    600

    Response elements

    Element

    Type

    Description

    Example

    object

    RequestId

    string

    The request ID.

    473469C7-AA6F-4DC5-B3DB-A3DC0D****3E
    DataFlowId

    string

    The data flow ID.

    df-194433a5be31****

    Examples

    Success response

    JSON format

    {
  "RequestId": "473469C7-AA6F-4DC5-B3DB-A3DC0D****3E",
  "DataFlowId": "df-194433a5be31****"
}

    Error codes

    HTTP status code

    Error code

    Error message

    Description
    400 IllegalCharacters The parameter contains illegal characters. The parameter contains illegal characters.
    400 MissingFsetId FsetId is mandatory for this action. The Fileset ID parameter is not specified.
    400 MissingSourceStorage SourceStorage is mandatory for this action. The SourceStorage parameter is not specified.
    400 MissingThroughput Throughput is mandatory for this action. The Throughput parameter is not specified.
    400 MissingFileSystemId FileSystemId is mandatory for this action. The FileSystemId parameter is not specified.
    400 InvalidFilesystemVersion.NotSupport This Api does not support this fileSystem version. This Api does not support this fileSystem version.
    400 DataFlow.Bucket.RegionUnmatched The bucket and file system are not in the same region. The bucket is inconsistent with the filesystem region.
    400 Oss.Path.Conflict The OSS directory conflicts and is occupied by other data flow tasks. The OSS directory conflicts and is occupied by other data flow tasks.
    403 OperationDenied.DependencyViolation The operation is denied due to dependancy violation. The conditions for this operation are not met.
    403 OperationDenied.NoStock The operation is denied due to no stock.
    403 OperationDenied.DependFset The operation is denied due to invalid fileset state. Failed to create the dataflow because the fileset is in an abnormal state.
    403 OperationDenied.ConflictOperation The operation is denied due to a conflict with an ongoing operation. The dataflow cannot be created because it conflicts with the current file system operation.
    403 OperationDenied.DependMountpoint The operation is denied because no mount point is found. Failed to create the dataflow because the file system does not have a mount target.
    403 OperationDenied.FsetAlreadyInUse The Fset is already bound to another data flow. The Fset is already in use.
    403 OperationDenied.AutoRefreshNotSupport The operation is denied. Auto refresh is not supported. The file system does not support automatic updates.
    403 OperationDenied.DependBucketTag The operation is denied. The OSS Bucket tag cpfs-dataflow is missing. Failed to create the dataflow because the OSS bucket does not have the cpfs-dataflow tag.
    403 OperationDenied.DataFlowNotSupported The operation is not supported. The file system does not support dataflow.
    403 InvalidOperation.DeletionProtection The operation is not allowed due to resource is protected by deletion protection. The instance has deletion protection enabled and cannot be released through the console or API.
    403 DataFlow.Bucket.AccessDenied Bucket access denied. Data flow authentication error.
    404 InvalidFileSystem.NotFound The specified file system does not exist. The specified file system does not exist.
    404 InvalidThroughput.OutOfBounds Throughput is out of bounds. The specified Throughput is not within the valid range.
    404 InvalidDescription.InvalidFormat Description format is invalid. The specified description has an invalid format.
    404 InvalidRefreshPath.InvalidParameter Refresh path is invalid. The path of the specified auto-update directory is invalid.
    404 InvalidRefreshPath.Duplicated Refresh path is duplicated. The path of the specified auto-update directory is duplicated.
    404 InvalidRefreshPath.NotFound Refresh path does not exist. The specified auto-update directory does not exist.
    404 InvalidRefreshPolicy.InvalidParameter Refresh policy is invalid. The auto-update policy is invalid.
    404 InvalidRefreshInterval.OutOfBounds Refresh interval is out of bounds. The specified RefreshInterval is not within the valid range.
    404 InvalidSourceStorage.Unreachable Source storage cannot be accessed. The source storage is inaccessible.
    404 InvalidSourceStorage.NotFound Source storage is not found. The specified source storage does not exist.
    404 InvalidSourceStorage.NotSupport Source storage type is not supported. The type of the specified source storage is not supported.
    404 InvalidSourceStorage.PermissionDenied The source storage access permission is denied. You do not have the access permissions for the source storage. Check the service-linked role authorization and the access tags of the source storage.
    404 InvalidSourceStorage.InvalidRegion Source storage region is invalid. The region of the source storage is abnormal. Check whether the source storage and CPFS are in the same region.
    404 InvalidSourceStorage.InvalidParameter Source storage has invalid parameters. The specified source storage path is invalid.
    404 InvalidSourceSecurityType.NotSupport The source security type is not supported. The security protection type of the specified source storage is not supported.
    404 InvalidAutoRefresh.TooManyAutoRefreshes The number of auto refreshes exceeds the limit. The number of specified automatic update configurations exceeds the limit.
    404 InvalidSourceStorage.NeedVersioning Source storage must enable versioning. The versioning feature must be enabled for the source storage.
    404 InvalidFsetId.NotFound The specified Fileset ID does not exist. The specified fileset does not exist.
    404 DataFlow.Bucket.NotExist Bucket does not exist. The bucket does not exist.

    See Error Codes for a complete list.

    Release notes

    See Release Notes for a complete list.