Container Service for Kubernetes:Expand a dynamically provisioned NAS volume

If you use Container Network File System (CNFS) to mount a NAS volume, the subdirectory quota feature is automatically enabled for the volume. For more information, see Use CNFS to automatically expand NAS volumes.

Limits

• Statically provisioned NAS volumes cannot be expanded.

• The subdirectory quota feature is supported only by NAS volumes that are used to mount General-purpose NAS file systems. In addition, the file systems must be mounted by using subdirectories. If you use NAS volumes to mount Extreme NAS file systems, you cannot enable the subdirectory quota feature for the volumes.

• For a single file system, you can configure quotas for a maximum of 500 directories. The maximum directory depth is eight levels. For example, the depth of the root directory / is zero level. The depth of the /workspace directory is one level. The depth of the /workspace/dir1 directory is two levels.

  Important

  • If you create a restrictive quota for a directory and the quota is used up, data can no longer be written to the directory. When the quota is used up, you can no longer increase file sizes, create files or directories, or move files to another directory. In addition, an IOError error occurs at the application layer. For information about how to resolve the issue, see Why is the "Disk quota exceeded" error message returned when data is written to a file system?

  • To prevent potential security risks, we recommend that you evaluate and test restrictive quotas before you create the restrictive quotas for core business-related directories.

  • After a quota is created, the quota is automatically initialized and is in the Initializing state. The initialization may take several hours or even longer to complete. The duration of the initialization process depends on the number of files and subdirectories in the directory of your file system. After the quota is initialized, the quota is in the Running state. You can query the status of a directory quota by using the NAS console or by calling the DescribeDirQuotas API operation.

  • The operation to enable a restrictive quota is asynchronously performed at the backend and requires 5 to 15 minutes to take effect.

Procedure

Step 1: Create a StorageClass that enables the subdirectory quota feature

You can enable the subdirectory quota feature when you create a StorageClass.

1. Create a file named alicloud-nas-quota-sc.yaml and copy the following content to the file. You can use the file to create a StorageClass that is used to mount a NAS volume and has the subdirectory quota feature enabled. After you enable the subdirectory quota feature, you can expand the NAS volume provisioned by using the StorageClass.

   Note

   To enable the subdirectory quota feature, set allowVolumeExpansion or volumeCapacity to true. If you set allowVolumeExpansion to true, the volumeCapacity parameter does not take effect. In this case, the subdirectory quota feature remains enabled.

   apiVersion: storage.k8s.io/v1
   kind: StorageClass
   metadata:
     name: alicloud-nas-quota-sc
   mountOptions:
     - nolock,tcp,noresvport
     - vers=3
   parameters:
     volumeAs: subpath  # Set the value to subpath. 
     server: "xxx.cn-hangzhou.nas.aliyuncs.com"
     archiveOnDelete: "false"
     path: "/abc"
     volumeCapacity: "true" # The quota feature is enabled for a volume that is dynamically provisioned based on the StorageClass. 
   provisioner: nasplugin.csi.alibabacloud.com
   reclaimPolicy: Delete
   allowVolumeExpansion: true # Enable the subdirectory quota feature to allow expansion of dynamically provisioned volumes.

   Parameter	Description
   mountOptions	Configure the options parameter and specify the NFS version in the mountOptions field. The version of the NFS protocol. We recommend that you use NFSv3. Extreme NAS file systems support only NFSv3. For more information about the NFS protocol, see NFS.
   volumeAs	You must set the value to subpath, which indicates that a subdirectory of the NAS file system is mounted as a persistent volume (PV).
   server	The mount target of the NAS file system if you mount a subdirectory of the NAS file system as a PV. For more information about how to query the domain name of a mount target, see Manage mount targets.
   archiveOnDelete	Specifies whether to delete the backend storage when reclaimPolicy is set to Delete. NAS is a shared storage service. You must specify both the reclaimPolicy and archiveOnDelete parameters to ensure data security. Default value: true.
   path	The subdirectory of the NAS file system that is mounted. If you mount an Extreme NAS file system, the path must start with /share.
   volumeCapacity	Specifies whether to enable the quota feature. Valid values: true and false. Note If allowVolumeExpansion is set to true, this parameter does not take effect.
   provisioner	The type of the storage driver that is used to provision the volume. In this example, the parameter is set to nasplugin.csi.alibabacloud.com. This indicates that the CSI plug-in provided by Alibaba Cloud is used.
   reclaimPolicy	The reclaim policy of the PV Valid values: Retain: retains the backend storage when the PV and persistent volume claim (PVC) are deleted. The backend storage may be a NAS file system. Delete: automatically deletes the backend storage and PV when the PVC is deleted.
   allowVolumeExpansion	This parameter is available only for General-purpose NAS file systems. If you set this parameter to true, a quota is configured for the PV that is dynamically provisioned by using the StorageClass.

2. Run the following command to create the StorageClass:

   kubectl apply -f alicloud-nas-quota-sc.yaml

Step 2: Create an application that has a dynamically provisioned volume mounted

1. Create a file named nas-sts.yaml and copy the following content to the file. The file is used to create a StatefulSet.

   The StorageClass created in the preceding step is specified in the following YAML temple. In addition, a PVC is automatically created after you deploy the following template.

   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
     name: nas-sts
   spec:
     selector:
       matchLabels:
         app: busybox
     serviceName: "busybox"
     replicas: 1
     template:
       metadata:
         labels:
           app: busybox
       spec:
         containers:
         - name: busybox
           image: mirrors-ssl.aliyuncs.com/busybox:latest
           command:
           - sleep
           - "3600"
           volumeMounts:
           - name: nas-pvc
             mountPath: /data
     volumeClaimTemplates:
     - metadata:
         name: nas-pvc
       spec:
         accessModes: [ "ReadWriteMany" ]
         storageClassName: "alicloud-nas-quota-sc"
         resources:
           requests:
             storage: 20Gi

2. Run the following command to create an application and mount the PVC to the application:

   kubectl apply -f nas-sts.yaml

Step 3: Verify the quota limit by writing data to the mount directory

Perform the following operations to verify the quota limit by writing data to the dynamically provisioned NAS volume you mounted in the preceding step. The quota for data stored in the volume is 20 GiB. When the size of data in the volume reaches 20 GiB, the system prompts that the disk quota is exceeded if you attempt to write data to the volume.

1. Run the following command to write 20 GiB of data to the /data directory that is mounted to the StatefulSet created in Step 2.

   dd if=/dev/zero of=20G.txt bs=1M count=10000

2. Wait 5 to 15 minutes and then check the quota details of the subdirectory in the NAS console.

   1. Log on to the NAS console.

   2. In the left-side navigation pane, choose File System > File System List.

   3. Find the file system that you want to manage and click the icon in the Actions column. Then, click Quota Management.

   4. On the Quota Management page, click Manage Quotas in the Actions column.

      The following figure shows that the subdirectory has a quota limit of 20 GiB. The used storage capacity is 19 GiB.

      When the 20-GiB storage capacity is exhausted, the Disk quota exceeded error message is prompted if you attempt to write more data to the subdirectory.

      If the quota is reached, you can manually expand the volume without interrupting the application.

Step 4: Manually expand the volume without interrupting the application

You can expand the volume by modifying the PVC bound to the PV. After you modify the PVC, you can check the actual quota to verify that the volume is expanded.

1. Expand the volume.

   kubectl patch pvc nas-pvc-nas-sts-0 -p '{"spec":{"resources":{"requests":{"storage":"30Gi"}}}}'

   Important

   The quota of a NAS directory is measured in GiB. After you modify the PVC, the CSI plug-in adjusts the quota of the mounted NAS directory to the new volume capacity specified by the PVC. During the adjustment, the CSI plug-in rounds up the capacity value to the nearest integer.

2. View information about the PV.

   kubectl get pv

   Expected output:

   NAME                              CAPACITY  ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                       STORAGECLASS         REASON     AGE
   nas-63c37cc2-b21e-4b56-b26f-****   30Gi          RWX          Delete        Bound    default/nas-pvc-nas-sts-0    alicloud-nas-quota-sc           25m23s

3. View information about the PVC.

   kubectl get pvc

   Expected output:

   NAME               STATUS   VOLUME                         CAPACITY  ACCESS MODES   STORAGECLASS            AGE
   nas-pvc-nas-sts-0  Bound   nas-63c37cc2-b21e-4b56-b26f-****   30Gi      RWX        alicloud-nas-quota-sc   25m10s

   The output shows that the quota of the subdirectory of the NAS volume is changed from 20 GiB to 30 GiB.

References

• For more information about NAS directory quotas, see Manage directory quotas.

• After you configure a quota for a NAS directory, you can monitor the usage of the NAS volume by using the node_volume_capacity_bytes_used metric. For more information about how to configure alert rules for volume capacity monitoring, see Create an alert rule for a Prometheus instance.

• If you cannot access NAS volumes due to permission issues, see Why does the system prompt chown: Operation not permitted when I mount a NAS volume?

• If you want to ensure data security among different users and user groups, refer to How do I enable user isolation or user group isolation in the NAS file system?

• For more information about read and write access issues when you use NAS, see FAQ about read and write access to files.

• If issues occur when you use NAS volumes mounted to containers, see FAQ about NAS volumes.