You can define expansion policies for Apsara File Storage NAS (NAS) volumes that are managed by Container Network File System (CNFS). This allows you to automatically expand NAS volumes when the storage usage exceeds a specific threshold. This topic describes how to use CNFS to automatically expand NAS volumes.
- A Kubernetes cluster is created. The Container Storage Interface (CSI) plug-in is
used as the volume plug-in. For more information, see Create an ACK managed cluster.
- If you want to use a new cluster, select the CSI volume plug-in and Dynamically Provision Volumes by Using the Default NAS File Systems and CNFS when you create the cluster.
- If you do not select Dynamically Provision Volumes by Using the Default NAS File Systems and CNFS when you create the cluster, you can use CNFS to manage NAS file systems. For more information, see Use CNFS to manage NAS file systems.
- The versions of csi-plugin and csi-provisioner are 1.20.5-ff6490f-aliyun or later. For more information about how to upgrade csi-plugin and csi-provisioner, see Upgrade CSI-Plugin and CSI-Provisioner.
- The default
dnsPolicyof csi-plugin is
ClusterFirst. If you want to use CNFS to automatically expand NAS volumes, you must change
- The version of storage-operator is 184.108.40.206-2aa33ba-aliyun or later. For more information about how to upgrade storage-operator, see Manage system components.
- A kubectl client is connected to the cluster. For more information, see Connect to ACK clusters by using kubectl.
Step 1: Configure an auto expansion policy
- View the status of the CNFS object. Make sure that the CNFS object is in the Available state.
- Run the following command to query the CNFS object:
kubectl get cnfs
NAME AGE default-cnfs-nas-837d6ea-20210819155623 14d
- Run the following command to view the status of the CNFS object:
kubectl get cnfs default-cnfs-nas-837d6ea-20210819155623 -o yaml | grep Available
- Run the following command to query the CNFS object:
- Use the following YAML template to create an auto expansion policy for a NAS volume.
cat << EOF | kubectl apply -f - apiVersion: storage.alibabacloud.com/v1alpha1 kind: StorageAutoScalerPolicy metadata: name: hybrid-expand-policy spec: pvcSelector: matchLabels: app: nginx # You must specify the label that is added to the persistent volume claim (PVC) and Deployment to which the NAS volume is mounted. In this example, app: nginx is specified. namespaces: - default - nginx conditions: - name: condition1 key: volume-capacity-used-percentage operator: Gt values: - "80" actions: - name: action1 type: volume-expand params: scale: 100% limits: 500Gi EOF
Parameter Description pvcSelector The label that is used to match the expansion policy with the PVC that is used to mount the NAS volume. In this example, nginx is used. namespaces The namespace of the PVC that is used to mount the NAS volume. If multiple namespaces are specified, the logical operator between the namespaces is OR. Default value: default. In this example, default and nginx are specified. conditions The conditions that trigger the action. The logical operator between multiple conditions is AND. Each condition includes the following parameters:
- key: specifies the type of the metric.
- volume-capacity-used-percentage: specifies that the storage usage threshold is expressed in percentage.
- operator: specifies the operator, which can be Gt (greater than), Lt (less than), Eq (equal to), or Ne (not equal to). The parameter value is not case-sensitive.
- values: specifies the threshold value.
In the example, the condition specifies that the action is triggered when the storage utilization of the PVC exceeds 80%.
actions Specifies the actions that are performed when the preceding conditions are met. Multiple actions are allowed. The following parameters are included:
- type: specifies the type of action. Valid value: volume-expand.
- scale: specifies the size of the storage that you want to add to the volume. Valid units: GiB and percentage (%).
- limits: specifies the maximum capacity of the PVC if the action is performed.
If multiple actions are specified in the actions section, the first action whose conditions are met is performed. The other actions are skipped.
In this example, action1 specifies that if the conditions are met, the NAS volume is expanded by 100%. The NAS volume can be expanded to at most 500 GiB.
- Use the following YAML template to create a PVC and a Deployment. Notice The auto expansion policy is applied to the PVC and Deployment that have the specified label. In this example, pvcSelector.matchLabels is set to
app: nginxin the auto expansion policy. In this case, the policy applies to the PVC and Deployment whose labels parameter is set to
cat << EOF | kubectl apply -f - kind: PersistentVolumeClaim apiVersion: v1 metadata: name: cnfs-nas-pvc labels: app: nginx # You must specify the same value as the pvcSelector.matchLabels parameter in the policy YAML template. In this example, app: nginx is specified. spec: accessModes: - ReadWriteMany storageClassName: alibabacloud-cnfs-nas resources: requests: storage: 50Gi --- apiVersion: apps/v1 kind: Deployment metadata: name: cnfs-nas-deployment labels: app: nginx spec: selector: matchLabels: app: nginx # You must specify the same value as the pvcSelector.matchLabels parameter in the policy YAML template. In this example, app: nginx is specified. template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.14.2 volumeMounts: - mountPath: "/data" name: cnfs-nas-pvc volumes: - name: cnfs-nas-pvc persistentVolumeClaim: claimName: cnfs-nas-pvc EOF
Step 2: Verify that the NAS volume can be automatically expanded
- Run the following command to query all pods in the default namespace:
kubectl get pod
NAME READY STATUS RESTARTS AGE cnfs-nas-deployment-56dbcc7fb7-wh79z 1/1 Running 0 20m
- Run the following command to log on to the cnfs-nas-deployment-56dbcc7fb7-wh79z application:
kubectl exec cnfs-nas-deployment-56dbcc7fb7-wh79z -ti sh
- Run the following command to write 50 GiB of data into the directory where the NAS
volume is mounted:
cd /data dd if=<data path> of=<mounted path>
- Run the following command to check events related to volume expansions:
kubectl get eventsExpected output:
default 0s Warning NotEnoughDiskSpace persistentvolumeclaim/cnfs-nas-pvc Pvc cnfs-nas-pvc is not enough disk space, namespace: default, totalSize:50Gi, usedSize:49Gi, usedPercentage:98.00%, threshold:85.00% default 1s Warning StartExpand persistentvolumeclaim/cnfs-nas-pvc Start to expand of pvc cnfs-nas-pvc from 50Gi to 100Gi, usedCapacityPercentage:98%, freeSize:1024MB. default 0s Normal Resizing persistentvolumeclaim/cnfs-nas-pvc External resizer is resizing volume nas-1acba306-e626-46f3-8441-110c10a6**** default 0s Warning ExternalExpanding persistentvolumeclaim/cnfs-nas-pvc Ignoring the PVC: didn't find a plugin capable of expanding the volume; waiting for an external controller to process this PVC. default 0s Normal FileSystemResizeRequired persistentvolumeclaim/cnfs-nas-pvc Require file system resize of volume on node default 0s Normal FileSystemResizeSuccessful pod/cnfs-nas-deployment-56dbcc7fb7-wh79z MountVolume.NodeExpandVolume succeeded for volume "nas-1acba306-e626-46f3-8441-110c10a6****"
The preceding output indicates that the storage usage of the NAS volume exceeds 80%, which triggers a volume expansion. The NAS volume is expanded from 50 GiB to 100 GiB.
- Check the monitoring information about CNFS in the CSI Nodes dashboard.
- Log on to the ACK console.
- In the left-side navigation pane of the ACK console, click Clusters.
- On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
- In the left-side navigation pane of the cluster details page, choose .
- On the Prometheus Monitoring page, click the CSI Nodes tab.
- On the CSI Nodes tab, set StorageType and Namespace based on the NAS volume, and select the PVC that is used to mount NAS volume. Then,
you can view the capacity of the NAS volume in the Total Capacity chart. In this example, set StorageType to nas, Namespace to default, and Pvc to cnfs-nas-pvc.
The preceding chart shows that the expansion occurred at 16:55:30 and the NAS volume is expanded to 100 GiB.