All Products
Search
Document Center

Container Service for Kubernetes:Configure a shared NAS volume

Last Updated:May 08, 2024

Apsara File Storage NAS (NAS) is a file system service intended for computing services, such as Elastic Compute Service (ECS) instances, Elastic High Performance Computing (E-HPC) clusters, and Container Service for Kubernetes (ACK) clusters. NAS provides seamless integration, storage sharing, and security management. NAS is suitable for multi-cluster applications because the ECS instances, E-HPC clusters, or ACK clusters where an application is deployed may need to access the same data source. This topic describes how to configure a shared NAS volume.

Background information

To ensure data security and share training data, we recommend that you configure and use shared volumes in the runtime environment where you submit jobs with Arena. This ensures that code and data are not deleted together with containers. We recommend that you use a shared volume to share data and code to the members in your team.

When you use Arena to submit jobs, if you have declared a shared volume and the path of the runtime environment to be mounted, you can specify the --data parameter to mount the shared volume to the path. This enables your jobs to reuse the data and code in the path.

In Kubernetes, volumes are typically declared by using persistent volumes (PVs) and persistent volume claims (PVCs). As the administrator of a Kubernetes cluster, you must create a PVC for each data scientist in your team. For example, you can mount the PVCs of User A and User B to the same NAS or CPFS file system, and mount these PVCs to different subdirectories to isolate User A from User B.

Step 1: Create a NAS file system

For more information about how to create a NAS file system, see Create a General-purpose NAS file system in the NAS console.

Note

Set the parameters based on the following descriptions:

  • Select General Purpose NAS as the file system type.

  • Select the region in which your ACK cluster is deployed.

  • Select the virtual private cloud (VPC) in which your ACK cluster is deployed.

  • Select NFS as the protocol type.

Step 2: Mount the file system

After the NAS file system is created, you need to mount the NFS file system to an ECS instance in the VPC of the ACK cluster and verify the mounting. This example shows how to mount the NFS file system in the console. For more information about how to mount the NFS file system by using other methods, see Usage notes.

Mount a NAS file system

Log on to the NAS console and perform the following operations:

  1. In the left-side navigation pane, choose File System > File System List.

  2. In the top navigation bar, select a region.

  3. Find the file system that you created and click Mount in the Actions column.

    Note

    The first time you use the mount feature in the NAS console, you must assign the AliyunServiceRoleForNasEcsHandler service-linked role to NAS. Follow the instructions in the dialog box to complete authorization. For more information, see Service-linked roles of NAS.

  4. In the Mount panel, perform the configurations to mount the file system.

    1. Select a mount target and click Next.

    2. Select an ECS instance and click Next.

      The selected ECS instance must reside in the same VPC as the file system. Otherwise, NAS prompts that the network may be disconnected. Proceed with caution.

      Note
    3. Mount the file system on the ECS instance.

      Parameter

      Description

      Mount Path

      Enter the local path on which you want to mount the file system. The local path resides on the Linux ECS instance.

      Automatic Mount

      By default, Automatic Mount at Startup is selected. When you restart the ECS instance, you do not need to re-mount the file system.

      Protocol Type

      Select the protocol type of the file system.

      • General-purpose NAS file systems support NFSv3 and NFSv4. If you do not need to modify the same NAS file on multiple ECS instances at a time, we recommend that you select NFSv3 to ensure optimal performance.

      • Extreme NAS file systems support only NFSv3.

      NAS Directory

      Enter the directory of the NAS file system.

      You can enter the root directory / or a subdirectory, for example, /abc.

      Note

      If the directory does not exist, you can select Confirm New Directory to create a directory in the file system. In addition, you must configure the user identifier (UID), group identifier (GID), and POSIX permissions for creating the directory. Otherwise, the default values and default permissions are used to create the directory.

      Mount Parameters

      We recommend that you use the default mount parameters. For more information, see the mount parameters described in Mount an NFS file system on a Linux ECS instance.

  5. Click Mount.

    The mount takes about 1 to 2 minutes. If the mount status is Mounted, the NAS file system is mounted on the ECS instance. If the mount status is Failed, you can use the troubleshooting tool to troubleshoot the issue. For more information, see Run the auto-check script to troubleshoot the failure of mounting an NFS file system on Linux.

Verify the mounting

  1. After you mount the file system to the ECS instance, you can use the file system in the same way as you use a local directory.

  2. Remotely connect to the ECS instance by referring to Connection methods, and run the following commands to access the file system:

    mkdir /mnt/dir1
    mkdir /mnt/dir2
    touch /mnt/file1
    echo 'some file content' > /mnt/file2
    ls /mnt
  3. If an output similar to the following example appears, you have accessed the General-purpose Capacity NFS file system.

    image.png

Note
  • Before you mount a file system on an ECS instance, create a mount target for the file system. For more information about how to add mount targets and view the addresses of mount targets, see Manage mount targets.

  • Pay attention to the following items when you add a mount target:

    • Select VPC as the type of mount target.

    • Select the same VPC and same vSwitches that are used by the specified ACK cluster.

Step 3: Create a PV and a PVC for the specified ACK cluster

Create a PV

  1. In the left-side navigation pane of the details page, choose Volumes > Persistent Volume Claims.

  2. In the upper-right corner of the Persistent Volume Claims page, click Create.

  3. In the Create PVC dialog box, set the following parameters.

    Parameter

    Description

    PVC Type

    You can select Cloud Disk, NAS, or OSS. In this example, NAS is selected.

    Name

    The name of the PVC. The name must be unique in the cluster.

    Allocation Mode

    In this example, Existing Volumes is selected.

    Note

    If no PV is created, you can set Allocation Mode to Create Volume and set the required parameters to create a PV. For more information, see Step 1: Create a PV.

    Existing Storage Class

    Click Select PV. Find the PV that you want to use and click Select in the Actions column.

    Capacity

    The capacity of the PV.

    Note

    The capacity claimed by the PVC cannot exceed the capacity of the PV that is bound to the PVC.

    Access Mode

    The default value is ReadWriteMany. You can also set the value to ReadWriteOnce or ReadOnlyMany.

  4. Click Create.

    After the PVC is created, you can view the PVC in the PVCs list. The PVC is bound to the corresponding PV.

Create a PVC

  1. In the left-side navigation pane of the details page, choose Volumes > Persistent Volume Claims.

  2. In the upper-right corner of the Persistent Volume Claims page, click Create.

  3. In the Create PVC dialog box, set the following parameters.

    Parameter

    Description

    PVC Type

    You can select Cloud Disk, NAS, or OSS. In this example, NAS is selected.

    Name

    The name of the PVC. The name must be unique in the cluster.

    Allocation Mode

    In this example, Existing Volumes is selected.

    Note

    If no PV is created, you can set Allocation Mode to Create Volume and set the required parameters to create a PV. For more information, see Step 1: Create a PV.

    Existing Storage Class

    Click Select PV. Find the PV that you want to use and click Select in the Actions column.

    Capacity

    The capacity of the PV.

    Note

    The capacity claimed by the PVC cannot exceed the capacity of the PV that is bound to the PVC.

    Access Mode

    The default value is ReadWriteMany. You can also set the value to ReadWriteOnce or ReadOnlyMany.

  4. Click Create.

    After the PVC is created, you can view the PVC in the PVCs list. The PVC is bound to the corresponding PV.

Step 4: Upload data to the NAS file system

The ACK cluster accesses shared data (the NAS file system created in Step 1) by using the PVC. Therefore, you only need to upload data to the NAS file system specified in the PVC.

  1. Use Workbench to log on to an ECS instance of the ACK cluster. For more information, see Connect to a Linux instance by using a password or key. For more information about how to connect to an ECS instance by using other methods, see Connection method overview.

  2. Mount the NFS file system in Step 2 to the /mnt directory of the ECS instance. Run the following commands to create the tf_data/ and pytorch_data/ directories in the /mnt directory to store the TF mnist and Pytorch mnist datasets, respectively:

    cd /mnt/
    mkdir tf_data/
    mkdir pytorch_data/
  3. Run the following command to download the TF mnist dataset:

    cd tf_data
    git clone https://code.aliyun.com/xiaozhou/tensorflow-sample-code.git
    mv tensorflow-sample-code/data/* ./ && rm -rf tensorflow-sample-code
  4. Run the following command to download the Pytorch mnist dataset:

    cd pytorch_data
    git clone https://code.aliyun.com/370272561/mnist-pytorch.git
    mv mnist-pytorch/MNIST ./ && rm -rf mnist-pytorch