Cloud Parallel File Storage:CPFS AI-Computing Edition supports the VSC high-speed access channel

What is VSC?

Virtual Storage Channel (VSC) is a high-performance storage access technology developed by Alibaba Cloud. It uses the RDMA protocol to provide compute nodes with low-latency, high-throughput access to file systems.

Key features

• RDMA passthrough access: Unlike the traditional TCP/IP stack, VSC minimizes CPU overhead and significantly reduces end-to-end latency.

• High throughput: Maximizes file system performance. Single-node throughput scales with instance specifications.

• Flexible compute-storage relationship: Manage the attachment between an instance and a file system with attach and detach operations, without configuring a network environment. A single VSC channel can attach to multiple file systems simultaneously.

For detailed performance metrics, including throughput, IOPS, and latency, see Product Specifications.

Typical use cases

• Reading and writing checkpoints for large model training

• Loading models for inference services

• High-concurrency data preprocessing

• Other I/O-sensitive AI computing tasks

VSC resource model

You manage the VSC high-speed access channel as a resource in the console and through OpenAPI.

• One compute node corresponds to one VSC high-speed access channel.

• One VSC high-speed access channel can attach to multiple CPFS AI-Computing Edition file systems.

VSC channel management follows the principle of "compute manages the channel, storage manages attachments":

VSC and VPC access comparison

CPFS AI-Computing Edition supports both VSC and VPC access methods. VSC is the recommended method; use VPC for supplementary access across storage-compute affinity domains.

Conceptual differences:

• VSC: Establishes a direct attach relationship between an instance and a storage resource without a network layer abstraction, resulting in lower performance overhead.

• VPC: First establishes a virtual private cloud, then connects compute and storage resources to the network. Access control relies on Access Control Lists (ACLs) and security groups.

Recommendation: When your compute nodes and CPFS AI-Computing Edition file system are in the same storage-compute affinity domain, use VSC high-speed access for optimal performance.

Determining VSC compatibility

A compute node can use VSC to access a CPFS AI-Computing Edition file system only if both are in the same storage-compute affinity domain. To verify compatibility:

1. Query the list of HPNZones associated with the file system: Call the CPFS API operation DescribeFileSystemAssociatedHpnZones. Pass the file system ID to retrieve a list of Lingjun cluster IDs (HPNZones) that support VSC access for that file system.

2. Identify the HPNZone of the compute node: Check the Lingjun cluster ID of your compute node in the Lingjun or PAI console.

3. Compare the results:

   • If the HPNZone of the compute node is in the list, you can use VSC high-speed access.

   • If the HPNZone of the compute node is not in the list, this indicates access across storage-compute affinity domains. You must use VPC access, which requires you to create a VPC mount target.

Note: A single file system can be associated with multiple HPNZones. VSC support does not require an exact HPNZone match. Always refer to the list returned by the DescribeFileSystemAssociatedHpnZones API operation.

APIs

The following CPFS OpenAPI operations manage VSC high-speed access channel and file system associations:

Note: The creation and deletion of a VSC high-speed access channel are managed through the OpenAPI of the respective compute product. For more information, see the documentation for that compute product.