Cloud Parallel File Storage (CPFS) for Lingjun is seamlessly integrated with Alibaba Cloud Platform for AI (PAI). You can configure a CPFS for Lingjun in PAI modules, such as Deep Learning Containers (DLC), Data Science Workshop (DSW), and Elastic Algorithm Service (EAS). This can be used as a dataset to store files and result files for deployment and training. This topic describes how to mount a CPFS for Lingjun dataset in a PAI module.
Prerequisites
A CPFS for Lingjun file system is created. For more information, see Create a file system.
When you prepare single-tenant Lingjun resources for the CPFS for Lingjun file system, you only need to configure CPFS for Lingjun on the CPFS tab of the Basic Parameters of Software Instance when creating a cluster. After the configuration is complete, the Lingjun cluster is automatically initialized and CPFS for Lingjun file system is mounted during automated deployment. For more information, see Create a Lingjun cluster with ACK activated.
Create a CPFS for Lingjun dataset
You do not need to create a dataset when you mount a CPFS for Lingjun in EAS. However, when you mount a CPFS for Lingjun dataset in DLC and DSW, you need to create a basic dataset in advance. The following table describes the key parameters.
Parameter | Description | |
Dataset configuration | Storage type | Select CPFS for LINGJUN. |
Import configuration | File system | Configure the ID of the CPFS for Lingjun file system you create. Log on to the NAS console and view the ID of the CPFS for Lingjun file system in the region where you want to create a cluster or from the dropdown list. |
File system path | The mount path of the CPFS for Lingjun file system. For example, set mount path to | |
Default mount path | The mount path of the CPFS for Lingjun file system that is mounted in DLC and DSW. For example, set mount path to | |
Mount a CPFS for Lingjun dataset
After a dataset is created, read data from a CPFS for Lingjun dataset that is mounted in DLC, DSW, or EAS.
When you mount CPFS using the following three methods, ensure that the CPFS instance and the Lingjun GPU are in the same zone and cluster.
In DLC
When you create a distributed training job in DLC, configure a CPFS for Lingjun file system in the Environment Information section. The following table describes the key parameters.
Parameter | Description |
Data Set | Click Custom Dataset, select a CPFS for Lingjun dataset and set the Mount Path parameter. When you run a DLC job, the system accesses the data from the CPFS for Lingjun file system in this path. Trigger Read-only to configure read and write permissions for custom datasets. |
Directly Mount | Only tasks that use Lingjun resources can use the CPFS for Lingjun file system whose name starts with BMCPFS. Click BMCPFS and configure parameters for the file system, and then configure a mount path to mount the dataset to the specified path of the DLC container. When you run a DLC job, the system accesses the data from the CPFS for Lingjun file system in this path. Trigger Read-only to configure read and write permissions for custom datasets. |
If a task uses general computing resources, you must configure virtual private cloud (VPC) that is the same as that of the CPFS for Lingjun. Otherwise, the submitted DLC training jobs may remain in the preparing environment for a long time. If a CPFS for Lingjun file system does not access to a VPC, add a VPC mount target and configure the VPC information.
In DSW
When you create a DSW instance, configure a CPFS for Lingjun dataset. The following table describes the key parameters.
Parameter | Description |
Custom Dataset | Click |
Mount Path | After a dataset is selected, the mount path of the dataset is displayed in the Mount Path area. In the DSW development environment, use this path to access data from the CPFS for Lingjun file system. |
, select a dataset of the CPFS for Lingjun that you create.If you select general computing resources for the Resource Quota parameter, you must configure a VPC that is the same as that of the CPFS for Lingjun. If a CPFS for Lingjun file system does not access to a VPC, add a VPC mount target and configure the VPC information.
In EAS
If you use Lingjun resources to deploy an EAS service, mount a CPFS for Lingjun file system. The following describes the key parameters:
Configure Model Settings in the Environment Information section, select CPFS for Lingjun as the mount type, and configure the following parameters:
Select a file system: Select a Cloud Parallel File Storage (CPFS) file system of your Alibaba Cloud account.
File System Path: the CPFS path where the files are stored.
Mount Path: the mount path of the service instance. The mount path is used to read files from the CPFS file system.
References
To create a dataflow between a source OSS bucket of Alibaba Cloud Account B and a CPFS for Lingjun file system of Alibaba Cloud Account A, see CPFS for Lingjun dataflow (invitational preview).
To manage the storage capacity of a directory of a file system and file quantity limit, see Fileset quota management.
To understand the performance of the file system, see Data monitoring.