Container Service for Kubernetes:Use CNFS to automatically expand NAS volumes

Step 1: Configure the storage-operator component and enable automatic expansion

The default storage-auto-expander in the storage-operator component is responsible for automatically expanding storage resources. Connect to the cluster and run the following command to modify the ConfigMap file of the storage-operator and enable the automatic expansion feature:

kubectl patch configmap/storage-operator \
  -n kube-system \
  --type merge \
  -p '{"data":{"storage-auto-expander":"{\"imageRep\":\"acs/storage-auto-expander\",\"imageTag\":\"\",\"install\":\"true\",\"template\":\"/acs/templates/storage-auto-expander/install.yaml\",\"type\":\"deployment\"}"}}'

Step 2: Configure an automatic expansion policy

1. View the status of the CNFS object. Make sure that the CNFS object is in the Available state.

   1. Query the CNFS object.

      kubectl get cnfs

      Expected output:

      NAME                                      AGE
      default-cnfs-nas-837d6ea-20210819155623   14d

   2. View the status of the CNFS object.

      kubectl get cnfs <The CNFS object queried in the previous step> -o yaml | grep Available

      Expected output:

      status: Available

2. Use the following YAML template to create an auto expansion policy for a NAS volume.

   cat << EOF | kubectl apply -f -
   apiVersion: storage.alibabacloud.com/v1alpha1
   kind: StorageAutoScalerPolicy
   metadata:
     name: hybrid-expand-policy
   spec:
     pvcSelector:
       matchLabels:
         app: nginx      # You must specify the label that is added to the persistent volume claim (PVC) and the Deployment to which the NAS volume is mounted. In this example, app: nginx is specified. 
     namespaces:
       - default
       - nginx
     conditions:
       - name: condition1
         key: volume-capacity-used-percentage
         operator: Gt
         values:
           - "80"
     actions:
       - name: action1
         type: volume-expand
         params:
           scale: 100%
           limits: 500Gi
   EOF

   Parameter	Description
   pvcSelector	The label that is used to match the expansion policy with the PVC that is used to mount the NAS volume. In this example, nginx is used.
   namespaces	The namespace of the PVC that is used to mount the NAS volume. If multiple namespaces are specified, the logical operator between the namespaces is OR. Default value: default. In this example, default and nginx are specified.
   conditions	The conditions that trigger the action. The logical operator between multiple conditions is AND. Each condition includes the following parameters: key: the type of the metric. volume-capacity-used-percentage: specifies that the storage usage threshold is expressed in percentage. operator: the operator, which can be Gt (greater than), Lt (less than), Eq (equal to), or Ne (not equal to). The parameter value is not case-sensitive. values: the threshold value. In the example, the condition specifies that the action is triggered when the storage utilization of the PVC exceeds 80%.
   actions	The actions that are performed when the preceding conditions are met. Multiple actions are allowed. Each action includes the following parameters: type: the type of the action. Only volume-expand is supported. scale: the size of the storage that you want to add to the volume. Valid units: GiB and percentage (%). limits: the maximum capacity of the PVC if the action is performed. If multiple actions are specified in the actions section, the first action whose conditions are met is performed. The other actions are skipped. In this example, action1 specifies that if the conditions are met, the NAS volume is expanded by 100%. The NAS volume can be expanded to up to 500 GiB.

3. Use the following YAML template to create a PVC and a Deployment.

   Important

   The auto expansion policy is applied to the PVC and the Deployment that have the specified label. In this example, pvcSelector.matchLabels is set to app: nginx in the auto expansion policy. In this case, the policy applies to the PVC and the Deployment whose labels parameter is set to app: nginx.

   cat << EOF | kubectl apply -f -
   kind: PersistentVolumeClaim
   apiVersion: v1
   metadata:
     name: cnfs-nas-pvc
     labels:
       app: nginx   # You must specify the same value as the pvcSelector.matchLabels parameter in the YAML template of the expansion policy. In this example, app: nginx is specified. 
   spec:
     accessModes:
       - ReadWriteMany
     storageClassName: alibabacloud-cnfs-nas
     resources:
       requests:
         storage: 50Gi
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: cnfs-nas-deployment
     labels:
       app: nginx
   spec:
     selector:
       matchLabels:
         app: nginx   # You must specify the same value as the pvcSelector.matchLabels parameter in the YAML template of the expansion policy. In this example, app: nginx is specified. 
     template:
       metadata:
         labels:
           app: nginx
       spec:
         containers:
         - name: nginx
           image: nginx:1.14.2
           volumeMounts:
           - mountPath: "/data"
             name: cnfs-nas-pvc
         volumes:
         - name: cnfs-nas-pvc
           persistentVolumeClaim:
             claimName: cnfs-nas-pvc
   EOF

Step 3: Verify that the NAS volume can be automatically expanded

1. Query all pods in the default namespace.

   kubectl get pod

   Expected output:

   NAME                                                            READY   STATUS    RESTARTS   AGE
   cnfs-nas-deployment-56dbcc7fb7-wh79z   1/1      Running            0          20m

2. Log on to the application pod and write 50 GiB of data to the directory to which the NAS volume is mounted.

   cd /data
   dd if=<data path> of=<mounted path>
   kubectl exec -it cnfs-nas-deployment-56dbcc7fb7-wh79z -ti -- dd if=/dev/zero of=/data/test1 bs=1G count=50

3. Check events related to volume expansion.

   kubectl get events

   The following output indicates that the storage usage of the NAS volume exceeds 80%, which triggers a volume expansion. The NAS volume is expanded from 50 GiB to 100 GiB.

   12s         Warning   StartExpand                  persistentvolumeclaim/cnfs-nas-pvc          Start to expand of pvc cnfs-nas-pvc from 50Gi to 100Gi, usedCapacityPercentage:96%, freeSize:2048MB.
   12s         Normal    ExternalExpanding            persistentvolumeclaim/cnfs-nas-pvc          waiting for an external controller to expand this PVC
   12s         Normal    Resizing                     persistentvolumeclaim/cnfs-nas-pvc          External resizer is resizing volume nas-462db2b2-717d-44fe-b0b6-fb4db03a****
   12s         Normal    VolumeResizeSuccessful       persistentvolumeclaim/cnfs-nas-pvc          Resize volume succeeded

4. Check the volume usage information in the CSI NAS dashboard.

   1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

   2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose Operations > Prometheus Monitoring.

   3. On the Prometheus Monitoring page, choose Storage Monitoring > CSI NAS.

   4. On the CSI NAS tab, select the namespace where the NAS storage volume is located and the PVC. Then, you can view the capacity of the NAS volume in the total capacity section.

      In this example, set Namespace to default, and PVC to cnfs-nas-pvc.

      The preceding chart shows that the expansion occurred at 10:23:30 and the NAS volume is expanded to 100 GiB.