All Products
Search
Document Center

Container Service for Kubernetes:Dynamically reserve PVs to mount disks

Last Updated:Oct 31, 2024

In addition to static provisioning, you can use dynamic provisioning to configure the system to automatically reserve persistent volumes (PVs) based on persistent volume claims (PVCs) and StorageClasses. Dynamic provisioning enables automatic and on-demand volume creation for applications. This reduces the complexity of storage infrastructure configuration in your Container Service for Kubernetes (ACK) cluster.

Limits

  • Disks cannot be mounted to pods deployed in zones other than the zone where the disks reside.

  • Disks for which the multi-attach feature is disabled can be mounted only to one pod.

  • The type of disk must match the Elastic Compute Service (ECS) instance types that are used in your cluster before you can launch a pod. For more information about the matching rules between disk types and ECS instance types, see Instance families.

    Important

    If you change the billing method of an ECS instance in your cluster from pay-as-you-go to subscription, you cannot change the billing method of its disks to subscription. Otherwise, the disks cannot be mounted to the cluster. You can purchase storage capacity units (SCUs) to reduce storage costs. For more information about SCUs, see Overview of SCUs.

Prerequisites

Important
  • If your cluster contains nodes that use CentOS 7.9 images or other OS images whose Linux kernel versions are earlier than 4.9 and the xfs file system is used to persist data (the fstype parameter is set to xfs), do not update the CSI plug-in to 1.24.7 or later. Linux kernel versions earlier than 4.9 are incompatible with CSI 1.24.7 or later. In this case, if you update the CSI plug-in to 1.24.7 or later, the pods to which the xfs file system is mounted may fail to start up. Other file systems are not affected. To resolve this issue, submit a ticket.

  • When you update the CSI plug-in to 1.26.4, you need to update both csi-plugin and csi-provisioner.

Usage notes

  • When you deploy an application, the system automatically purchases a disk and provisions a PV based on the specified StorageClass. If you already have a disk, we recommend that you use a statically provisioned disk volume.

  • When you purchase a disk, the disk size that you specify cannot exceed the corresponding capacity range for individual disks. For more information, see Disk types.

  • When the system recreates a pod, the original disk mounted to the pod is remounted. If the system fails to schedule the pod to the original zone during the recreation, the pod remains in the Pending state.

  • Dynamic provisioning supports only pay-as-you-go disks. For more information about the billable items of disks, see Billing of block storage devices. For more information about the pricing of disks, visit the Elastic Compute Service product page.

You can create a StorageClass, PVC, and application in the ACK console or by using kubectl.

Use a dynamically provisioned disk volume in the ACK console

Step 1: Create a StorageClass

  1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

  2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose Volumes > StorageClasses.

  3. In the upper-right corner of the StorageClasses page, click Create.

  4. In the Create dialog box, configure the parameters of the StorageClass.

    The following table describes the key parameters.

    Parameter

    Description

    Name

    The name of the StorageClass.

    PV Type

    You can select Cloud Disk or NAS. In this example, Cloud Disk is selected.

    Parameter

    The type of the disk. By default, the type parameter is added and set to cloud_essd. Valid values of the type parameter:

    • cloud_efficiency: ultra disk.

    • cloud_auto: ESSD AutoPL disk.

    • cloud_ssd: standard SSD.

    • cloud_essd: ESSD.

    • cloud_essd_entry: ESSD Entry disk.

    You can specify one or more values. For example, you can set this parameter to type: cloud_efficiency, cloud_ssd, cloud_essd. The system attempts to create a disk of the specified types in sequence. The system stops trying once a disk is created.

    Note

    The disk types supported by an ECS instance vary based on the instance type. For more information, see FAQ about EBS.

    You can click Add to add custom parameters. The following custom parameters are supported:

    • resourceGroupId: the resource group to which the disk belongs. This parameter is optional. Default value: "".

    • regionId: the ID of the region where the disk is created. The value must be set to the region ID of the cluster. This parameter is optional.

    • zoneId: the ID of the zone where the disk is created. This parameter is optional.

      • If your cluster is deployed in a single zone, set the value to the ID of the zone.

      • If your cluster is deployed across zones, you can set zoneId to multiple zone IDs based on your business requirements. Example: cn-beijing-a, cn-beijing-b.

    • fstype: the file system of the disk. This parameter is optional. Valid values: ext4 and xfs. Default value: ext4.

    • mkfsOptions: the parameters that are specified to format the disk. This parameter is optional. Example: mkfsOptions: "-O project,quota".

    • diskTags: the custom tags of the disk. This parameter is optional. Example: diskTags: "a:b,b:c". You can also use the format diskTags/a: b (starting from v1.30.3).

    • encrypted: specifies whether to encrypt the disk. This parameter is optional. The default value is false, which indicates that the disk is not encrypted.

    • performanceLevel: the performance level (PL) of the ESSD. Valid values: PL0, PL1, PL2, and PL3. This parameter is optional. Default value: PL1. For more information, see Capacity and PLs.

    • volumeExpandAutoSnapshot: specifies whether to create a snapshot to back up the data stored in the disk before the disk is resized. This parameter is optional. Default value: closed. Valid values:

      • forced: cancels the disk resizing task if the system fails to create the snapshot.

      • besteffort: generates a warning event and resizes the disk if the system fails to create the snapshot.

      • closed: disables the automatic snapshot feature.

      If you have high requirements for data security, we recommend that you set the parameter to forced to prevent data loss that may occur when the system fails to resize the disk.

      Note

      This parameter applies only to ESSD resizing. The automatic snapshot is automatically deleted after the system resizes the disk and is retained for 24 hours if the system fails to resize the disk.

    • provisionedIops: the provisioned performance of the disk, such as the disk Input/Output Operations per Second (IOPS). This parameter is optional.

      Valid values: 0 to min{50,000, 1,000 × Capacity - Baseline IOPS}.

      Baseline IOPS = min{1,800 + 50 × Capacity, 50,000}.

      Note

      This parameter is supported only by ESSD AutoPL disks. For more information, see ESSD AutoPL disk specifications.

    burstingEnabled: specifies whether to enable performance burst for the disk. This parameter is optional. Default value: false. Valid values:

    • true: enables performance burst for the disk.

    • false: disables performance burst for the disk.

      Note

      This parameter is supported only by ESSD AutoPL disks. For more information, see ESSD AutoPL disk specifications.

    • multiAttach: This parameter is optional. Set it to true to enable multi-attach. For more information, see Multi-attach for cloud disks.

    Reclaim Policy

    The reclaim policy of the disk. Default value: Delete. Valid values:

    • Delete: When PVCs are deleted, the related PVs and disks are also deleted.

    • Retain: When PVCs are deleted, the related PVs and disks are retained. The PVs and disk data can only be manually deleted.

    If you have high requirements for data security, we recommend that you use the Retain policy to prevent data loss caused by user errors.

    Binding Mode

    The binding mode of the disk. Default value: Immediate. Valid values:

    • Immediate: creates a disk when the PVC that uses the StorageClass is created. This way, a disk is created and a PV is provisioned before you create a pod that uses the disk.

    • WaitForFirstConsumer: delays the binding and provisioning of a PV until a pod that uses the PVC is created. After the pod is scheduled to a node, a disk is created in the zone where the node is deployed and provisioned as the PV.

  5. After you complete the configuration, click Create.

    After the StorageClass is created, it is displayed in the StorageClasses list.

Step 2: Create a PVC

  1. In the left-side navigation pane of the details page, choose Volumes > Persistent Volume Claims.

  2. In the upper-right corner of the Persistent Volume Claims page, click Create.

  3. In the Create PVC dialog box, set the parameters of the PVC.

    Parameter

    Description

    PVC Type

    You can select Cloud Disk, NAS, or OSS. In this example, Cloud Disk is selected.

    Name

    The name of the PVC. The name must be unique in the namespace.

    Allocation Mode

    In this example, Use StorageClass is selected, which indicates that a StorageClass is used to dynamically provision a PV.

    Existing Storage Class

    Select an existing StorageClass to dynamically provision a PV.

    Capacity

    The capacity claimed by the PVC.

    Access mode

    You can use only ReadWriteOnce. For more information, see Use a dynamically provisioned disk volume by using kubectl.

  4. Click Create.

    After the PVC is created, the PVC is displayed in the PVCs list. The PVC is bound to a PV.

Step 3: Deploy an application

  1. In the left-side navigation pane of the details page, choose Workloads > StatefulSets.

  2. In the upper-right corner of the StatefulSets page, click Create from Image.

  3. Configure the application parameters.

    This example shows how to set the volume parameters. For more information about other parameters, see Use a StatefulSet to create a stateful application.

  4. You can configure local storage volumes and cloud storage volumes for an ACK cluster. In this example, Cloud Storage is selected. Mount the disk volume that you created to the /tmp path of the container. After the disk volume is mounted, the container data that is generated in the /tmp path is stored in the disk volume.数据卷

  5. Set other parameters and click Create.

    After the application is created, you can use the volume to store application data.

Use a dynamically provisioned disk volume by using kubectl

Step 1: Create a StorageClass

For multi-zone clusters, you can select the following methods to create StorageClasses based on the actual scenarios:

Delayed PVC-PV binding

The delayed PVC-PV binding mode is applicable when existing disks do not belong to the zone of the ECS instance that you want to use. In the following example, a StorageClass is created by using a file named storage-class-csi-wffc.yaml.

  1. Create a file named storage-class-wffc.yaml and copy the following code to the file:

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: alicloud-disk-wait-for-first-consumer
    provisioner: diskplugin.csi.alibabacloud.com
    parameters:
      type: cloud_auto,cloud_essd,cloud_ssd # The system attempts to create a disk of the specified types in sequence. The actual types of disks that are created depend on the instance types of the ECS instances to which the disks are mounted and the disk types supported by the zones where the ECS instances are deployed. 
      fstype: ext4
      diskTags: "a:b,b:c"
      encrypted: "false"
      performanceLevel: PL1
      volumeExpandAutoSnapshot: "forced" # This parameter takes effect only when the type parameter is set to cloud_essd. 
      provisionedIops: "40000"
      burstingEnabled: "false"
    volumeBindingMode: WaitForFirstConsumer
    reclaimPolicy: Retain
    allowVolumeExpansion: true

    Parameter

    Description

    metadata

    name

    The name of the StorageClass. The name must start with a lowercase letter, and can contain only lowercase letters, digits, periods (.), and hyphens (-).

    provisioner

    Set this parameter to diskplugin.csi.alibabacloud.com. This indicates that the provisioner plug-in for Alibaba Cloud disks is used to create the StorageClass.

    parameters

    For more information, see Parameter.

    volumeBindingMode

    The binding mode of the disk. Default value: Immediate. You can also set the value to WaitForFirstConsumer.

    • Immediate: creates a disk when the PVC that uses the StorageClass is created. This way, a disk is created and a PV is provisioned before you create a pod that uses the disk.

    • WaitForFirstConsumer: delays the binding and provisioning of a PV until a pod that uses the PVC is created. After the pod is scheduled to a node, a disk is created in the zone where the node is deployed and provisioned as the PV.

    reclaimPolicy

    The reclaim policy of the disk. Default value: Delete. You can also set the value to Retain.

    • Delete: When PVCs are deleted, the related PVs and disks are also deleted.

    • Retain: When PVCs are deleted, the related PVs and disks are retained. The PVs and disk data can only be manually deleted.

    If you have high requirements for data security, we recommend that you use the Retain policy to prevent data loss caused by user errors.

    allowVolumeExpansion

    If you set this parameter to true, the disk can be automatically resized.

  2. Run the following command to create the StorageClass:

    kubectl apply -f storage-class-wffc.yaml
  3. View the created StorageClass.

    1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

    2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose Volumes > StorageClasses.

Create a disk before creating a pod

  1. Create a file named storage-class-immediate.yaml and copy the following code to the file:

    apiVersion: storage.k8s.io/v1
    kind: StorageClass
    metadata:
      name: alicloud-disk-immediate
    provisioner: diskplugin.csi.alibabacloud.com
    parameters:
      type: cloud_essd,cloud_ssd,cloud_efficiency # The system attempts to create a disk of the specified types in sequence. The actual types of disks that are created depend on the instance types of the ECS instances to which the disks are mounted and the disk types supported by the zones where the ECS instances are deployed.
      regionId: cn-beijing
      zoneId: cn-beijing-b
      encrypted: "false"
    reclaimPolicy: Retain
    allowVolumeExpansion: true
    volumeBindingMode: Immediate

    Parameter

    Description

    metadata

    name

    The name of the StorageClass. The name must start with a lowercase letter, and can contain only lowercase letters, digits, periods (.), and hyphens (-).

    provisioner

    Set this parameter to diskplugin.csi.alibabacloud.com. This indicates that the provisioner plug-in for Alibaba Cloud disks is used to create the StorageClass.

    parameters

    For more information, see Parameter

    reclaimPolicy

    The reclaim policy of the disk. Default value: Delete. You can also set the value to Retain.

    • Delete: When PVCs are deleted, the related PVs and disks are also deleted.

    • Retain: When PVCs are deleted, the related PVs and disks are retained. The PVs and disk data can only be manually deleted.

    If you have high requirements for data security, we recommend that you use the Retain policy to prevent data loss caused by user errors.

    allowVolumeExpansion

    If you set this parameter to true, the disk can be automatically resized.

    volumeBindingMode

    The binding mode of the disk. Default value: Immediate. You can also set the value to WaitForFirstConsumer.

    • Immediate: creates a disk when the PVC that uses the StorageClass is created. This way, a disk is created and a PV is provisioned before you create a pod that uses the disk.

    • WaitForFirstConsumer: delays the binding and provisioning of a PV until a pod that uses the PVC is created. After the pod is scheduled to a node, a disk is created in the zone where the node is deployed and provisioned as the PV.

  2. Run the following command to create the StorageClass:

    kubectl apply -f storage-class-immediate.yaml
  3. View the created StorageClass.

    1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

    2. On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose Volumes > StorageClasses.

Step 2: Create a PVC

  1. Create a file named pvc-disk.yaml and copy the following code to the file:

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      name: disk-pvc
    spec:
      accessModes:
      - ReadWriteOnce
      volumeMode: Filesystem
      resources:
        requests:
          storage: 25Gi
      storageClassName: alicloud-disk-wait-for-first-consumer 

    Parameter

    Description

    name

    The name of the PVC.

    accessModes

    The access mode of the PVC.

    volumeMode

    The volume mode of the disk. Valid values: Filesystem and Block. This parameter is optional. Default value: Filesystem.

    storageClassName

    The name of the StorageClass that you want to associate with the PVC.

    storage

    The disk size claimed by the PVC. The minimum capacity is 20 GiB.

    The available access modes depend on the multiAttach parameter in the StorageClass and the volumeMode parameter in the PVC. The following table describes the specifications:

    Parameter setting

    Access mode

    multiAttach

    volumeMode

    ReadWriteOnce

    ReadOnlyMany

    ReadWriteMany

    image.png

    Any

    image.png

    image.png

    image.png

    image.png

    Filesystem

    image.png

    image.png

    image.png

    image.png

    Block

    image.png

    image.png

    image.png

  2. Run the following command to create the PVC:

    kubectl create -f pvc-disk.yaml
  3. View the created PVC.

    In the left-side navigation pane of the cluster details page in the ACK console, choose Volumes > Persistent Volume Claims. You can view the created PVC on the Persistent Volume Claims page.

Step 3: Create an application

  1. Run the following command to create a Secret for setting the root password of the MySQL application. Make sure to change the password if you are working in a production environment.

    kubectl create secret generic mysql-pass --from-literal=password=mypassword
  2. Create a file named mysql.yaml and copy the following code to the file:

    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: mysql
    spec:
      selector:
        matchLabels:
          app: mysql
      serviceName: mysql
      template:
        metadata:
          labels:
            app: mysql
        spec:
          containers:
          - name: mysql
            image: mysql:8
            env:
            - name: MYSQL_ROOT_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: mysql-pass
                  key: password
            ports:
            - containerPort: 3306
              name: mysql
            volumeMounts:
            - name: pvc-disk
              mountPath: /var/lib/mysql
          volumes:
            - name: pvc-disk
              persistentVolumeClaim:
                claimName: disk-pvc

    Parameter

    Description

    mountPath

    The path to which the disk is mounted.

    claimName

    The name of the PVC that is mounted to the application.

  3. Run the following command to create an application and mount the PVC to the application:

    kubectl create -f mysql.yaml
  4. View the application.

    In the left-side navigation pane of the cluster details page in the ACK console, choose Workloads > StatefulSets. You can view the created application on the StatefulSets page.

Test whether the dynamically provisioned disk volume can be used to persist data

After a pod is deleted and recreated, the pod still stores the data that is written before the pod is deleted. Perform the following steps to test whether data is persisted to the disk:

  1. Confirm that the disk is mounted in the MySQL application by running the following command to check whether a new disk is mounted to the /var/lib/mysql path:

    kubectl exec mysql-0 -- df -h /var/lib/mysql

    Expected output:

    Filesystem      Size  Used Avail Use% Mounted on
    /dev/vdd         25G  213M   25G   1% /var/lib/mysql
  2. Create a file in the disk.

    1. Run the following command to create a file named test-persistent in the /var/lib/mysql path:

      kubectl exec mysql-0 -- touch /var/lib/mysql/test-persistent
    2. Run the following command to verify that the file is created:

      kubectl exec mysql-0 -- ls /var/lib/mysql/test-persistent

      Expected output:

      /var/lib/mysql/test-persistent
  3. Run the following command to delete the pod named mysql-0:

    kubectl delete pod mysql-0

    Expected output:

    pod "mysql-0" deleted
  4. Test whether the file still exists after the pod is deleted.

    1. Run the following command to check whether the pod run as expected:

      kubectl get pod mysql-0

      Expected output:

      NAME      READY   STATUS    RESTARTS   AGE
      mysql-0   1/1     Running   0          12s
    2. Run the following command to query the created file:

      kubectl exec mysql-0 -- ls /var/lib/mysql/test-persistent

      Expected output:

      /var/lib/mysql/test-persistent

      The test-persistent file still exists. This indicates that data is persisted to the dynamically provisioned disk.

References