Container Service for Kubernetes:Use dynamically provisioned disk volumes for persistent storage

How it works

Dynamic disk provisioning with a StatefulSet follows three steps:

1. Define a template. Create a new StorageClass or use a default one. The template defines parameters such as disk type, performance level, and reclaim policy.

2. Declare storage requirements. Define volumeClaimTemplates in the StatefulSet and reference the StorageClass. This declares the PVC specifications each pod uses, such as storage capacity and access mode.

3. Automatic volume creation and mounting. When the StatefulSet creates a pod, the system generates a unique PVC for the pod based on the template. The Container Storage Interface (CSI) component creates a persistent volume (PV) based on the StorageClass rules, binds the PV to the PVC, and mounts the disk to the pod.

Scope and limitations

• Zone restrictions: All disk types except regional Enterprise SSDs (ESSDs) cannot be mounted across zones. They can be mounted only to pods in the same zone.

• Instance family restrictions: Some disk types can be attached only to specific instance families.

• CSI component requirements: The csi-plugin and csi-provisioner components must be installed. > Note: CSI components are installed by default. Make sure that you have not manually uninstalled them. Check the installation status on the Add-ons page. For the best experience, upgrade the CSI components to the latest version.

• Virtual node requirements: To use disks on virtual nodes, your cluster and kube-scheduler versions must meet the following requirements:

  Cluster version	kube-scheduler version
  1.28 or later	6.9.3 or later
  1.26	6.8.7
  1.24	6.4.7
  1.22	6.4.5

Select a StorageClass

ACK provides multiple default StorageClasses. A StorageClass cannot be modified after creation. If the defaults do not meet your requirements, create a new one. See Create a StorageClass manually.

Mount a disk to a StatefulSet

Use a StatefulSet when each pod replica needs its own independent disk.

1. Create a file named statefulset.yaml. The following example creates a StatefulSet with two pods. It uses volumeClaimTemplates to automatically create and bind independent persistent storage for each pod. > Important: Configuring securityContext.fsGroup in a pod causes the kubelet to recursively change file permissions (chmod/chown) when the volume is mounted. If the volume contains many files, this significantly increases the mount time. > > For clusters running Kubernetes 1.20 or later, set fsGroupChangePolicy to OnRootMismatch. This performs a recursive permission change only on the first mount and only if the permissions of the volume's root directory do not match the required permissions. If you need further performance tuning or more granular permission control, use an initContainer to run permission adjustment commands before the main application container starts.

      apiVersion: apps/v1
      kind: StatefulSet
      metadata:
        name: web
      spec:
        serviceName: "nginx"
        replicas: 2
        selector:
          matchLabels:
            app: nginx
        template:
          metadata:
            labels:
              app: nginx
          spec:
            # Configure the following securityContext to optimize mount performance
            securityContext:
              fsGroup: 1000
              fsGroupChangePolicy: "OnRootMismatch"
            containers:
            - name: nginx
              image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
              ports:
              - containerPort: 80
              volumeMounts:
              # Mount the data volume to the /data directory of the container
              # The name must be the same as metadata.name defined in volumeClaimTemplates
              - name: pvc-disk
                mountPath: /data
        # Define the PVC template
        volumeClaimTemplates:
        - metadata:
            name: pvc-disk
          spec:
            # Access mode
            accessModes: [ "ReadWriteOnce" ]
            # Associate the previously created StorageClass
            storageClassName: "alicloud-disk-wait-for-first-consumer"
            resources:
              requests:
                # Requested storage capacity, which is the disk size
                storage: 20Gi

2. Create the StatefulSet:

      kubectl create -f statefulset.yaml

3. Confirm that the pods are in the Running state:

      kubectl get pod -l app=nginx

4. Check the mount path and confirm that the disk is mounted. > Note: In this example, the pod name is web-1. Replace it with your actual pod name. Expected output:

      kubectl exec web-1 -- df -h /data

      Filesystem      Size  Used Avail Use% Mounted on
      /dev/vdb         20G   24K   20G   1% /data

Verify persistent storage

To verify that data persists after a pod is recreated, write data to the pod, delete the pod, and check whether the data remains.

1. Write test data to the pod. For the pod web-1, create a test file in the mounted disk path /data: Expected output:

      kubectl exec web-1 -- touch /data/test
      kubectl exec web-1 -- ls /data

      lost+found
      test

2. Simulate a pod failure by deleting the pod: Run kubectl get pod -l app=nginx again. A new pod named web-1 is automatically created.

      kubectl delete pod web-1

3. Verify the data in the new pod. Check the /data folder in the new pod web-1: The test file still exists, confirming that data persists after the pod is deleted and recreated.

      kubectl exec web-1 -- ls /data

      lost+found
      test

Mount a disk to a single pod or single-replica Deployment

For applications that do not require multi-replica scaling or a stable Network ID, manually create a PVC and mount it to a pod or Deployment.

The process: select a StorageClass, create a PVC, and mount the PVC in the application.

1. Prepare a StorageClass.

2. Create a PVC to request storage resources.

   kubectl

   1. Create a file named disk-pvc.yaml.

         apiVersion: v1
         kind: PersistentVolumeClaim
         metadata:
           name: disk-pvc
         spec:
           # Access mode
           accessModes:
           - ReadWriteOnce
           volumeMode: Filesystem
           resources:
             requests:
               # Requested storage capacity, which is the disk size
               storage: 20Gi
           # Associate with the previously created StorageClass
           storageClassName: alicloud-disk-topology-alltype

      The following table describes the parameters.

      Parameter	Description
      accessModes	The access mode of the volume. Valid values: ReadWriteOnce, ReadOnlyMany, ReadWriteMany. The supported values depend on the multiAttach setting in the StorageClass and the volumeMode setting in the PVC. multiAttach specifies whether to enable multi-attach for disks. Default: false. If multiAttach is false and volumeMode is set to any value, only ReadWriteOnce is supported. If multiAttach is true and volumeMode is Filesystem, only ReadWriteOnce and ReadOnlyMany are supported. If multiAttach is true and volumeMode is Block, all three access modes are supported. Important: In this scenario, the access mode is typically ReadWriteOnce (RWO), which means the volume can be mounted by only one pod at a time. The number of Deployment replicas cannot be greater than 1. If you scale out the Deployment, the new pods will be stuck in the Pending state because they cannot mount the disk that is already in use.
      volumeMode	The mode of the persistent volume. Valid values: Filesystem (default): The volume is formatted and mounted as a directory. Block: The volume is provided to the pod as an unformatted block device.
      storage	The requested storage capacity. The capacity range varies by disk type. Make sure the value of storage complies with the capacity limits of the disk type corresponding to the referenced StorageClass to prevent disk creation failures.
      storageClassName	The StorageClass to bind.

   2. Create the PVC:

         kubectl create -f disk-pvc.yaml

   3. View the PVC:

         kubectl get pvc

      Because the StorageClass uses the WaitForFirstConsumer mode, the PVC is in the Pending state until the first pod that uses it is successfully scheduled.

         NAME       STATUS    VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS                            VOLUMEATTRIBUTESCLASS   AGE
         disk-pvc   Pending                                      alicloud-disk-topology-alltype           <unset>                 14s

   Console

   1. In the navigation pane on the left of the cluster management page, choose Storage > Persistent Volume Claims.

   2. On the Persistent Volume Claims page, click Create. Set PVC Type to Cloud Disk and configure the parameters as prompted.

      Parameter	Description
      Allocation Mode	Select Use StorageClass.
      Existing StorageClass	A default or manually created StorageClass.
      Capacity	The requested storage capacity. The capacity range varies by disk type. Make sure the value of storage complies with the capacity limits of the disk type corresponding to the referenced StorageClass to prevent disk creation failures.
      Access Mode	Only ReadWriteOnce is supported. This means the volume can be mounted as read-write by a single pod.

      After the PVC is created, you can view it on the Persistent Volume Claims page.

3. Mount the PVC in an application.

   1. Create a file named disk-deployment.yaml.

         apiVersion: apps/v1
         kind: Deployment
         metadata:
           name: single-pod-app
         spec:
           # Make sure the number of replicas is 1
           replicas: 1
           selector:
             matchLabels:
               app: nginx-single
           template:
             metadata:
               labels:
                 app: nginx-single
             spec:
               containers:
               - name: nginx
                 image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
                 ports:
                 - containerPort: 80
                 # Define the mount point in the container
                 volumeMounts:
                 - name: my-persistent-storage  # Must match the name defined in volumes below
                   mountPath: /data  # Mount to the /data directory in the container
               # Declare and reference the PVC at the pod level
               volumes:
               - name: my-persistent-storage # The volume to be referenced by the container
                 persistentVolumeClaim:
                   claimName: disk-pvc # Reference the previously created PVC

   2. Deploy the Deployment:

         kubectl create -f disk-deployment.yaml

4. Verify the mount result.

   1. Verify that the pod is running:

         kubectl get pods -l app=nginx-single

   2. Log on to the pod and check whether the disk is mounted to the /data directory:

         # Get the pod name
         POD_NAME=$(kubectl get pods -l app=nginx-single -o jsonpath='{.items[0].metadata.name}')
         # Run the df -h command
         kubectl exec $POD_NAME -- df -h /data

      The following output indicates that the 20 GiB disk is successfully mounted:

         Filesystem      Size  Used Avail Use% Mounted on
         /dev/vdb         20G   24K   20G   1% /data

Going live

Billing

Disks dynamically created using a StorageClass are billed on a pay-as-you-go basis. For more information, see Elastic Block Storage billing and Elastic Block Storage pricing.

FAQ

References

• If you encounter issues when you use disk volumes, see Disk volume FAQ.

• For optimization suggestions for multi-zone disk deployments, see Recommended configurations for high availability of disk volumes.

• If your cluster still uses the deprecated FlexVolume component, migrate FlexVolume to CSI.

• For more information about how to create a workload, see Create a StatefulSet and Create a Deployment.

• If you no longer use a disk and want to stop billing for it, release the disk. Releasing a disk deletes the disk and its data, and stops billing.