CPFS AI Edition supports multiple pods mounting the same file system simultaneously in ReadWriteMany mode, enabling shared access to training data and models. This topic describes how to mount a CPFS AI Edition file system as a static storage volume in containers by using the ACK CSI plug-in.
Before you begin
An ACK Lingjun managed cluster has been created. The cluster runs Kubernetes 1.26 or later and uses Alibaba Cloud Linux 3 as the node operating system.
A CPFS AI Edition file system has been created. Record the file system ID. For more information, see Create a CPFS AI Edition file system.
The following storage components are installed in the ACK cluster. You can check the versions on the Component Management page of the cluster or Manage CSI components:
Component
Required version
csi-plugin / csi-provisioner
≥ v1.33.1
cnfs-nas-daemon
≥ 0.1.2
bmcpfs-csi-controller / bmcpfs-csi-node
Managed by ACK. Manual installation is not required.
If a pod is scheduled to a non-Lingjun node or a Lingjun node in a different compute-storage affinity zone, you must create a VPC mount target in advance and record the mount target domain name (in the format of
cpfs-***-vpc-***.<Region>.cpfs.aliyuncs.com). The VPC to which the VPC mount target belongs must be the same as the VPC of the ACK cluster. For more information, see Manage VPC mount points.
Considerations
When you mount a CPFS file system by using VSC, the node on which the pod runs must be in the same compute-storage affinity zone as the CPFS AI Edition file system instance.
If you use CSI container-based mounting for CPFS, you must select the option to support high-performance file storage access when you create a Lingjun node group.
Before a Lingjun node is taken offline due to a fault, you must first drain pods from the node. Otherwise, the cluster metadata becomes inconsistent, which causes pod resources to remain and fail to be reclaimed properly.
Mounting different subdirectories from the same CPFS instance through multiple PVs in a single pod is not supported. Due to underlying driver limitations, this configuration causes the pod to fail to start.
We recommend that you create only one PV/PVC for the CPFS instance, and then use the
subPathfield in the podvolumeMountsconfiguration to mount the required subdirectories separately.subPathis implemented based on a lightweightbind mountmechanism and does not introduce additional performance overhead.
Access channels and scheduling
In an ACK Lingjun cluster, the network channel through which a pod accesses CPFS AI Edition is determined by the pod scheduling location. The CSI plug-in automatically selects the optimal channel:
Pod scheduling location | Access channel | Typical scenario |
Lingjun GPU node in the same compute-storage affinity zone | VSC + RDMA (optimal performance) | GPU training tasks |
Lingjun node in a different compute-storage affinity zone | Automatically switches to VPC mount target (TCP) | Cross-zone training or inference tasks |
Non-Lingjun node (CPU node pool) | VPC mount target (TCP) | CPU pods: data preprocessing, inference services, scheduling management, etc. |
To enable automatic channel switching, you must configure both mountpointAutoSwitch: "true" and vpcMountTarget (VPC mount target domain name) in the volumeAttributes of the PV. For configuration details, see Step 1: Create a PV and PVC below.
To determine whether a Lingjun node and a CPFS file system are in the same compute-storage affinity zone, see Overview of mount access.
Step 1: Create a PV and PVC
-
Save the following template as
bmcpfs-pv-pvc.yaml, replacing placeholders with your file system ID and (if needed) VPC mount target.PV parameters
Parameter
Description
Required
accessModesAccess mode for the PV.
Yes
capacity.storageDeclared storage capacity. Does not affect actual capacity.
Yes
csi.driverDriver type. For CPFS for Lingjun, use
bmcpfsplugin.csi.alibabacloud.com.Yes
csi.volumeHandleID of the CPFS for Lingjun file system.
Yes
csi.volumeAttributes.vpcMountTargetVPC mount target domain name. Required for non-Lingjun nodes; omit if pods schedule only to Lingjun nodes.
Conditional
csi.volumeAttributes.mountpointAutoSwitchEnables automatic switching between VSC and VPC mount points. Use with
vpcMountTarget.No
mountOptionsMount options.
No
PVC parameters
Parameter
Description
Required
accessModesAccess mode requested by the PVC. Must match the PV.
Yes
resources.requests.storageStorage capacity allocated to the pod. Must not exceed the PV capacity.
Yes
volumeModeMount mode. Set to
Filesystem.Yes
volumeNameName of the PV to bind to this PVC.
Yes
apiVersion: v1 kind: PersistentVolume metadata: name: bmcpfs spec: accessModes: - ReadWriteMany capacity: storage: 10Ti claimRef: name: bmcpfs namespace: default csi: driver: bmcpfsplugin.csi.alibabacloud.com volumeAttributes: # Required if pods schedule to non-Lingjun nodes or cross-zone automatic VPC switching is enabled. # Omit if pods schedule only to Lingjun nodes. vpcMountTarget: cpfs-***-vpc-***.<Region>.cpfs.aliyuncs.com # Enables automatic switching between VSC and VPC mount points. # Use together with vpcMountTarget. mountpointAutoSwitch: "true" # Replace with your CPFS for Lingjun file system ID. volumeHandle: bmcpfs-***** mountOptions: [] --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: bmcpfs namespace: default spec: accessModes: - ReadWriteMany resources: requests: storage: 10Ti volumeMode: Filesystem volumeName: bmcpfs -
Apply the configuration.
kubectl apply -f bmcpfs-pv-pvc.yaml -
Confirm that the PVC is bound to the PV.
kubectl get pvc bmcpfsExpected output:
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE bmcpfs Bound bmcpfs 10Ti RWX <unset> 51sThe
STATUSisBound, confirming the PVC is bound to the PV.
Step 2: Create an application and mount CPFS
Scenario 1: Mount the entire CPFS file system
Use this approach when all containers need full access to the CPFS file system.
-
Save the following YAML template as
cpfs-test.yaml.apiVersion: apps/v1 kind: Deployment metadata: name: cpfs-test labels: app: cpfs-test spec: replicas: 2 selector: matchLabels: app: cpfs-test template: metadata: labels: app: cpfs-test spec: containers: - name: nginx image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6 ports: - containerPort: 80 volumeMounts: - name: pvc-cpfs mountPath: /data volumes: - name: pvc-cpfs persistentVolumeClaim: claimName: bmcpfs -
Create the Deployment.
kubectl create -f cpfs-test.yaml -
Verify both pods are running.
kubectl get pod -l app=cpfs-testExpected output:
NAME READY STATUS RESTARTS AGE cpfs-test-76b77d64b5-2hw96 1/1 Running 0 42s cpfs-test-76b77d64b5-dnwdx 1/1 Running 0 42s -
Verify the CPFS volume mount inside a pod.
kubectl exec -it <pod-name> -- mount | grep /dataExpected output:
bindroot-f0a5c-******:cpfs-*******-vpc-****.cn-shanghai.cpfs.aliyuncs.com:/ on /data type fuse.aliyun-alinas-efc (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=1048576)The mount type
fuse.aliyun-alinas-efcat/dataconfirms the CPFS volume is mounted.
Scenario 2: Mount subdirectories for data isolation
In shared storage scenarios such as multi-tenant or multitasking setups, multiple containers can share one CPFS volume while isolating data in separate directories. Use volumeMounts.subPath to mount different subdirectories into different containers from the same PVC.
If asubPathdirectory (for example,workspace/alpha) does not exist, it is created automatically.
-
Save the following YAML template as
pod.yaml. This pod runs two containers, each mounting a different subdirectory of the same PVC (bmcpfs).apiVersion: v1 kind: Pod metadata: name: cpfs-subpath-demo-pod spec: containers: - name: task-alpha-container image: busybox:1.35 command: ["/bin/sh", "-c", "sleep 3600"] volumeMounts: - name: cpfs-storage mountPath: /data/workspace # Mount path inside the container subPath: workspace/alpha # Mount the workspace/alpha subdirectory only - name: task-beta-container image: busybox:1.35 command: ["/bin/sh", "-c", "sleep 3600"] volumeMounts: - name: cpfs-storage mountPath: /data/workspace # Mount paths can be identical across containers subPath: workspace/beta # Mount the workspace/beta subdirectory only volumes: - name: cpfs-storage persistentVolumeClaim: claimName: bmcpfs # Reference the PVC created earlier -
Deploy the pod.
kubectl apply -f pod.yaml -
Verify the mount and write permissions for
task-alpha-container.-
Connect to the container.
kubectl exec -it cpfs-subpath-demo-pod -c task-alpha-container -- /bin/sh -
Confirm that the CPFS volume is mounted.
df -hExpected output (shared directory mounted at
/data/workspace):Filesystem Size Used Available Use% Mounted on ... 192.XX.XX.0:/share 10.0T 1.0G 10.0T 0% /data/workspace ...The filesystem entry at
/data/workspaceconfirms the CPFS subdirectory is mounted. -
Check the parent directory structure.
ls -l /data/Expected output:
total 4 drwxr-xr-x 2 root root 4096 Aug 15 10:00 workspace -
Write a test file and exit.
echo "hello from alpha" > /data/workspace/alpha.log exit
-
-
Verify the
task-beta-containermount and data isolation.-
Connect to the container.
kubectl exec -it cpfs-subpath-demo-pod -c task-beta-container -- /bin/sh -
Write a test file.
echo "hello from beta" > /data/workspace/beta.log -
List files in the mount point.
ls -l /data/workspace/Expected output:
total 4 -rw-r--r-- 1 root root 16 Aug 15 10:05 beta.logbeta.logis present butalpha.logis absent, confirming data isolation between containers.
-
Step 3: Verify the mount
kubectl exec -it <pod-name> -- mount | grep /data
# Expected output contains fuse.aliyun-alinas-efc, indicating the EFC + FUSE protocol is in use