Disk volumes are suitable for applications that require high IOPS and low latency but do not require data sharing. If no cloud disk exists within your Alibaba Cloud account, you can create a dynamically provisioned disk volume to automate disk creation and mounting. This allows you to meet the requirements for data persistence. This topic describes how to use a dynamically provisioned disk volume and how to verify that a dynamically provisioned disk volume can be used to persist data.
Use scenarios
Disks are suitable for the following scenarios:
You want to create applications that require high disk I/O throughput and do not require data sharing. The applications can use storage services such as MySQL and Redis.
You want to write logs at high speeds.
You want to persist data in a way that is independent of the pod lifecycle.
In addition to static volume provisioning based on existing disks, Kubernetes supports dynamic volume provisioning based on StorageClasses. Dynamic provisioning allows you to dynamically create and allocate storage resources. After you associate a StorageClass with a persistent volume claim (PVC), the system automatically creates and binds a persistent volume (PV) based on the configurations of the PVC and StorageClass. Dynamic provisioning automates the creation and configuration of storage resources to meet the storage requirements of your applications. Dynamic provisioning also reduces the complexity of storage infrastructure orchestration.
Prerequisites
The Container Storage Interface (CSI) plug-in is installed in your cluster.
In the left-side navigation pane of the cluster management page, choose Update csi-plugin and csi-provisioner.
. On the Storage tab, you can check whether csi-plugin and csi-provisioner are installed. For more information about how to update CSI plug-ins to use specific capabilities, seeIf your cluster uses FlexVolume, you must migrate the cluster to the CSI plug-in because FlexVolume is no longer available. For more information, see Upgrade from FlexVolume to CSI.
Usage notes
Disks cannot be shared. If multi-attach is not enabled for disks, each disk can be mounted to only one pod. For information about the multi-attach feature, see Use the multi-attach and NVMe reservation features of NVMe disks.
You can mount a disk only to a pod that resides in the same zone as the disk.
When a pod is rebuilt, the original disk is mounted to the pod. If the pod cannot be scheduled to the original zone due to specific limits, the pod remains in the Pending state because the disk cannot be attached.
We recommend that you mount disks to pods or StatefulSets instead of Deployments.
NoteIf multi-attach is disabled, a disk can be mounted to only one pod. If you want to mount a disk to a Deployment, you must set the number of pod replicas to one for the Deployment. If you configure multiple pods, you cannot mount a separate disk volume to each pod. In addition, you cannot specify the volume mount and unmount priorities of pods. When you restart a pod in a Deployment, the disk may fail to be mounted to the restarted pod due to the update policy used by Deployments. We recommend that you do not mount disks to Deployments.
If the application configuration includes the
securityContext.fsgroup
parameter when you use a disk volume, kubelet automatically runs thechmod
andchown
commands after the volume is mounted, which may slow down the volume mount process.NoteAfter you add the
securityContext.fsgroup
parameter to the application configuration, Container Service for Kubernetes (ACK) automatically modifies the ownership of files in the volume when the disk is mounted to the application. The time required for ownership modification varies based on the number of files in the volume. If a large number of files exist in the volume, the modification process require a long period of time. For clusters that run Kubernetes 1.20 or later, you can set thefsGroupChangePolicy
parameter in the pod configuration toOnRootMismatch
. This ensures that ACK modifies file ownership only during the first time the pod is started. When you update or recreate the pod after creation, the volume mount process does not involve ownership modification. If the preceding setting does not meet your business requirements, we recommend that you create init containers and grant the init containers the permissions to perform relevant operations.
StorageClasses
A StorageClass defines the class of storage that you want to use. You can specify different parameters in a StorageClass to automate storage resource supply and adjustment based on the parameters. For more information, see Storage basics.
After you install the CSI plug-in, ACK provides default StorageClasses for cloud disks. If the default StorageClasses does not meet your business requirements, you can create a custom StorageClass:
You cannot modify the properties of an existing StorageClass. If you want to update StorageClass properties, you must create a new StorageClass with the properties that you want to use. You can view the configurations of a StorageClass in the ACK console or by running the
kubectl describe sc <storageclass-name>
command. For more information about StorageClass parameters, see StorageClass parameters.Enterprise SSDs (ESSDs) in cloud boxes must be of performance level 0 (PL0). If you use a cloud box, you must set
performanceLevel
toPL0
when you configure a StorageClass for ESSDs. The default StorageClass for ESSDs is used to create PL1 ESSDs. If you use a cloud box, you must create a new StorageClass for ESSDs.Standard SSDs, ultra disks, and basic disks are cloud disks of the previous generation and are unavailable for purchase in specific regions and zones. We recommend that you use ESSDs at performance level 0 (PL0 ESSDs) or ESSD Entry disks instead of ultra disks and basic disks and use ESSD AutoPL disks instead of standard SSDs.
Default StorageClass | Disk category |
alicloud-disk-topology-alltype | High availability (HA) configurations. When you use the StorageClass to mount a disk to a pod, the system creates a disk in the zone where the Elastic Compute Service (ECS) that hosts the pod resides. Inventory must be sufficient for the disk category in the zone, and the disk category must be supported by the instance type of the ECS instance that hosts the pod. The system first attempts to create an ESSD. If the system fails to create an ESSD, it attempts to create a standard SSD. If the system fails to create a standard SSD, it attempts to create an ultra disk. Important If you specify |
alicloud-disk-essd | Creates an ESSD. The default PL is PL1. |
alicloud-disk-ssd | Creates a standard SSD. |
alicloud-disk-efficiency | Creates an ultra disk. |
Disks that are created by using dynamic provisioning based on StorageClasses use the pay-as-you-go billing method.
For more information about the billing rules of cloud disks, see Block storage devices.
For more information about the prices of disks, see Prices of block storage devices.
Mount a dynamically provisioned disk volume by using kubectl
Step 1: Create a StorageClass
If the default StorageClasses does not meet your business requirements, you can create a custom StorageClass: You cannot modify the properties of an existing StorageClass. If you want to update StorageClass properties, you must create a new StorageClass with the properties that you want to use.
You cannot mount a cloud disk to a node that resides in a zone different from the disk. In addition, the ECS instance types to which a cloud disk can be mounted depend on the category of the disk. When you mount a dynamically provisioned disk volume to a pod, make sure that the disk and the node that hosts the pod reside in the same zone. In addition, the ECS instance type of the node must be supported by the category of the disk. For more information about the matching rules between disk categories and ECS instance types, see Overview of instance families.
Connect to your cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster or Manage Kubernetes clusters via kubectl in Cloud Shell.
Create a file named disk-sc.yaml and copy the following template to the file:
NoteIf your cluster is deployed across multiple zones, we recommend that you use delayed PVC-PV binding to prevent mounting failures caused by zone inconsistency between the cloud disk and ECS node.
WaitForFirstConsumer mode (delayed PVC-PV binding)
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: alicloud-disk-wait-for-first-consumer provisioner: diskplugin.csi.alibabacloud.com parameters: type: cloud_auto,cloud_essd,cloud_ssd # The system attempts to create a disk based on the priorities of the specified disk categories. The actual category of the disk that is created depends on the instance type of the ECS instance to which the disk is mounted and the disk categories supported by the zone where the ECS instance is deployed. fstype: ext4 diskTags: "a:b,b:c" encrypted: "false" performanceLevel: PL1 # If you use a cloud box, you must set the value to PL0. provisionedIops: "40000" burstingEnabled: "false" volumeBindingMode: WaitForFirstConsumer reclaimPolicy: Retain allowVolumeExpansion: true
Immediate mode
apiVersion: storage.k8s.io/v1 kind: StorageClass metadata: name: alicloud-disk-immediate provisioner: diskplugin.csi.alibabacloud.com parameters: type: cloud_essd,cloud_ssd,cloud_efficiency # The system attempts to create a disk based on the priorities of the specified disk categories. The actual category of the disk that is created depends on the instance type of the ECS instance to which the disk is mounted and the disk categories supported by the zone where the ECS instance is deployed. regionId: cn-beijing zoneId: cn-beijing-b encrypted: "false" reclaimPolicy: Retain allowVolumeExpansion: true volumeBindingMode: Immediate
The following table describes the parameters in the StorageClass.
Parameter
Description
name
The name of the StorageClass. The name must start with a lowercase letter, and can contain only lowercase letters, digits, periods (.), and hyphens (-).
provisioner
The type of the driver. In this example, the parameter is set to
diskplugin.csi.alibabacloud.com
, which indicates that the CSI plug-in provided by Alibaba Cloud is used.parameters
The
type
parameter is required. This parameter specifies the disk categories that are used. Valid values of the type parameter:cloud_auto
: ESSD AutoPL disk.cloud_essd
: ESSD. This is the default value.cloud_essd_entry
: ESSD Entry disk.cloud_ssd
: standard SSD.cloud_efficiency
: ultra disk.
You can set this parameter to any combination of the preceding values. For example, you can set this parameter to
type: cloud_ssd,cloud_essd,cloud_auto
. In this case, the system tries to create a disk in the specified order of the categories. The system stops trying once a disk is created.ImportantYou can specify disk categories based on the zones and instance types of the nodes in your cluster and your requirements for disk billing and performance. For more information, see the following topics:
volumeBindingMode
The binding mode of the disk. Default value:
Immediate
. You can also set the value toWaitForFirstConsumer
.Immediate
: creates a disk when the PVC that uses the StorageClass is created. This way, a disk is created and provisioned before you create a pod that uses the disk.WaitForFirstConsumer
: delays the binding and provisioning of a PV until a pod that uses the PVC is created. After the pod is scheduled to a node, a disk is created in the zone where the node is deployed and provisioned as the PV.
The system selects the zone in which the disk is created based on the volume binding mode:
If the
WaitForFirstConsumer
mode is used, the system creates the disk in the zone where the first pod that uses the PVC is deployed.If the
Immediate
mode is used, the system selects the zone in which the disk is created based on the value of thezoneId
parameter. If only one zone ID is specified in the zoneId parameter, the system creates the disk in the specified zone. If multiple zone IDs are specified in thezoneId
parameter, the system selects a zone from the specified zones in a round-robin manner.
NoteIf your cluster is deployed across multiple zones, we recommend that you use the
WaitForFirstConsumer
mode. For more information, see Recommended storage settings for cross-zone deployment.reclaimPolicy
The reclaim policy of the disk. Default value:
Delete
. You can also set the value toRetain
.Delete
: When the PVC is deleted, the PV and cloud disk are also deleted.Retain
: When the PVC is deleted, the PV and cloud disk are retained. You need to manually delete the PV and cloud disk.
If you have high requirements for data security, we recommend that you set the value to Retain to prevent accidental deletion.
allowVolumeExpansion
To enable auto disk volume expansion, set the value to
true
.Create a StorageClass.
kubectl create -f disk-sc.yaml
View the StorageClass.
kubectl get sc
Expected output:
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE alicloud-disk-efficiency diskplugin.csi.alibabacloud.com Delete Immediate true 158m alicloud-disk-essd diskplugin.csi.alibabacloud.com Delete Immediate true 158m alicloud-disk-ssd diskplugin.csi.alibabacloud.com Delete Immediate true 158m alicloud-disk-topology-alltype diskplugin.csi.alibabacloud.com Delete WaitForFirstConsumer true 158m alicloud-disk-wait-for-first-consumer diskplugin.csi.alibabacloud.com Retain WaitForFirstConsumer true 10s
Step 2: Create a PVC
Create a file named disk-pvc.yaml and copy the following content to the file:
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: disk-pvc spec: accessModes: - ReadWriteOnce volumeMode: Filesystem resources: requests: storage: 20Gi storageClassName: alicloud-disk-wait-for-first-consumer # In this example, a StorageClass that uses delayed PVC-PV binding is specified.
The following table describes the parameters.
Parameter
Description
accessModes
The access mode of the volume. For more information, see Volume access modes.
Valid values:
ReadWriteOnce
,ReadOnlyMany
, andReadWriteMany
. The supported access modes depend on themultiAttach
parameter of the StorageClass and thevolumeMode
parameter of the PVC.If the
multiAttach
parameter is set tofalse
and thevolumeMode
parameter is not empty, only theReadWriteOnce
access mode is supported.If the
multiAttach
parameter is set totrue
and thevolumeMode
parameter is set toFilesystem
, theReadWriteOnce
orReadOnlyMany
access mode is supported.If the
multiAttach
parameter is set totrue
and thevolumeMode
parameter is set toBlock
, all three volume modes are supported.
NoteThe
multiAttach
parameter is used to specify whether to enable the multi-attach feature. Default value: false. The default value disables the multi-attach feature. For more information, see Multi-attach for cloud disks.volumeMode
The mode of the volume. Valid values:
Filesystem
: the file system mode. This is the default value.Block
: the block device mode.
storage
The storage capacity that you want to allocate to the pod. This is also the capacity of the disk.
NoteFor more information about the capacity ranges supported by different disk categories, see Block storage performance.
storageClassName
The name of the StorageClass that you want to associate.
Create a PVC.
kubectl create -f disk-pvc.yaml
View the PVC.
kubectl get pvc
The output shows that the PVC is in the Pending state. In this case, no PV is created because the StorageClass uses delayed PVC-PV binding.
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE disk-pvc Pending alicloud-disk-wait-for-first-consumer <unset> 14s
Step 3: Create an application and mount a disk to the application
Create a file named disk-test.yaml and copy the following content to the file.
The following code block specifies the configuration of a StatefulSet that provisions one pod. The pod requests storage resources by using the
disk-pvc
PVC, which is mounted to the/data
path of the pod.apiVersion: apps/v1 kind: StatefulSet metadata: name: disk-test spec: replicas: 1 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6 ports: - containerPort: 80 volumeMounts: - name: pvc-disk mountPath: /data volumes: - name: pvc-disk persistentVolumeClaim: claimName: disk-pvc
Create a StatefulSet and mount a disk to the StatefulSet.
kubectl create -f disk-test.yaml
Check whether the pod provisioned by the StatefulSet is deployed.
kubectl get pod -l app=nginx
The following output shows that one pod is deployed for the StatefulSet.
NAME READY STATUS RESTARTS AGE disk-test-0 1/1 Running 0 14s
View files in the mount path to check whether the disk is mounted.
kubectl exec disk-test-0 -- df -h /data
Expected output:
Filesystem Size Used Avail Use% Mounted on /dev/vdb 20G 24K 20G 1% /data
Mount a dynamically provisioned disk volume in the ACK console
Step 1: Create a StorageClass
If the default StorageClasses does not meet your business requirements, you can create a custom StorageClass: You cannot modify the properties of an existing StorageClass. If you want to update StorageClass properties, you must create a new StorageClass with the properties that you want to use.
You cannot mount a cloud disk to a node that resides in a zone different from the disk. In addition, the ECS instance types to which a cloud disk can be mounted depend on the category of the disk. When you mount a dynamically provisioned disk volume to a pod, make sure that the disk and the node that hosts the pod reside in the same zone. In addition, the ECS instance type of the node must be supported by the category of the disk. For more information about the matching rules between disk categories and ECS instance types, see Overview of instance families.
Log on to the ACK console. In the left-side navigation pane, click Clusters.
On the Clusters page, find the cluster that you want to manage and click its name. In the left-side pane, choose .
On the StorageClasses page, click Create.
In the Create dialog box, configure the parameters and click Create.
Parameter
Description
Example
Name
Enter a custom name for the StorageClass. The name must follow the format requirements displayed in the ACK console.
alicloud-disk-wait-for-first-consumer
PV Type
Select Cloud Disk.
Cloud Disk
Parameter
By default, the
type
parameter is added. This parameter specifies the disk category. Valid values of the type parameter:cloud_auto
: ESSD AutoPL disk.cloud_essd
: ESSD. This is the default value.cloud_essd_entry
: ESSD Entry disk.cloud_ssd
: standard SSD.cloud_efficiency
: ultra disk.
You can set this parameter to any combination of the preceding values. For example, you can set this parameter to
type: cloud_ssd,cloud_essd,cloud_auto
. In this case, the system tries to create a disk in the specified order of the categories. The system stops trying once a disk is created.ImportantYou can specify disk categories based on the zones and instance types of the nodes in your cluster and your requirements for disk billing and performance. For more information, see the following topics:
type: cloud_auto,cloud_essd,cloud_ssd
Reclaim Policy
The reclaim policy of the disk. Default value:
Delete
. You can also set the value toRetain
.Delete
: When the PVC is deleted, the PV and cloud disk are also deleted.Retain
: When the PVC is deleted, the PV and cloud disk are retained. You need to manually delete the PV and cloud disk.
If you have high requirements for data security, we recommend that you set the value to Retain to prevent accidental deletion.
Retain
Binding Mode
The binding mode of the disk. Default value:
Immediate
. You can also set the value toWaitForFirstConsumer
.Immediate
: creates a disk when the PVC that uses the StorageClass is created. This way, a disk is created and provisioned before you create a pod that uses the disk.WaitForFirstConsumer
: delays the binding and provisioning of a PV until a pod that uses the PVC is created. After the pod is scheduled to a node, a disk is created in the zone where the node is deployed and provisioned as the PV.
The system selects the zone in which the disk is created based on the volume binding mode:
If the
WaitForFirstConsumer
mode is used, the system creates the disk in the zone where the first pod that uses the PVC is deployed.If the
Immediate
mode is used, the system selects the zone in which the disk is created based on the value of thezoneId
parameter. If only one zone ID is specified in the zoneId parameter, the system creates the disk in the specified zone. If multiple zone IDs are specified in thezoneId
parameter, the system selects a zone from the specified zones in a round-robin manner.
NoteIf your cluster is deployed across multiple zones, we recommend that you use the
WaitForFirstConsumer
mode. For more information, see Recommended storage settings for cross-zone deployment.WaitForFirstConsumer
After you create the StorageClass, you can view the StorageClass on the StorageClass page.
Step 2: Create a PVC
In the left-side navigation pane of the cluster details page in the ACK console, choose
.In the upper-right corner of the Persistent Volume Claims page, click Create.
In the Create dialog box, configure the parameters and click Create.
Parameter
Description
Example
PVC Type
Select Cloud Disk.
Cloud Disk
Name
Enter a custom name for the PVC. The name must follow the format requirements displayed in the ACK console.
diks-pvc
Allocation Mode
In this example, Use StorageClass is selected.
Use StorageClass
Existing Storage Class
Select the StorageClass that you want to associate with the PVC.
alicloud-disk-topology-alltype
Capacity
Specify the storage capacity that you want to allocate to the pod. This is also the capacity of the disk.
NoteFor more information about the capacity ranges supported by different disk categories, see Block storage performance.
20Gi
Access Mode
You must select ReadWriteOnce, which indicates that the volume is mounted only to one pod in read-write mode.
ReadWriteOnce
After you create the PVC, you can view the PVC on the Persistent Volume Claims page.
Step 3: Create an application and mount a disk to the application
In the left-side navigation pane of the details page, choose .
In the upper-right corner of the StatefulSets page, click Create from Image.
Configure the parameters of the StatefulSet and click Create.
The following table describes some of the parameters. Configure other parameters based on your business requirements. For more information, see Use a StatefulSet to create a stateful application.
Wizard page
Parameter
Description
Example
Basic Information
Name
Enter a custom name for the StatefulSet. The name must follow the format requirements displayed on the UI.
disk-test
Replicas
The number of pod replicas provisioned by the StatefulSet.
1
Container
Image Name
The address of the image used to deploy the application.
anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
Required Resources
Specify the number of vCores, the amount of memory, and the amount of ephemeral storage required by the application.
CPU: 0.25 vCore
Memory: 512 MiB
Ephemeral-Storage: Skip
Volume
Click Add PVC and configure the parameters.
Mount Source: Select the PVC that you created in Step 2.
Container Path: Specify the container path to which you want to mount the disk.
Mount Source: disk-pvc.
Container Path: /data
Check whether the application is deployed.
On the StatefulSets page, click the name of the application that you created.
On the Pods tab, check whether the pod is in the Running state.
Check whether data persistence is enabled based on the disk by using kubectl
The StatefulSet created in the preceding example provisions one pod, and a disk is mounted to the pod. If you delete the pod, the system automatically recreates the pod. The original disk is mounted to the new pod and data is retained on the disk. To test whether data is persisted to the disk, perform the following steps:
View files in the mount path to check whether files on the disk can be viewed.
kubectl exec disk-test-0 -- ls /data
Expected output:
lost+found
Create a file on the disk.
kubectl exec disk-test-0 -- touch /data/test
Delete the pod.
kubectl delete pod disk-test-0
NoteAfter you delete the pod, the system automatically recreates the pod.
Check the new pod.
kubectl get pod -l app=nginx
The following output shows that the new pod has the same name as the pod you deleted.
NAME READY STATUS RESTARTS AGE disk-test-0 1/1 Running 0 27s
Check whether the original disk is mounted to the pod and the file is retained on the disk.
kubectl exec disk-test-0 -- ls /data
The following output shows that the
test
file is retained on the disk.lost+found test
References
If errors occur when you use disk volumes, refer to FAQ about disk volumes.
For more information about how to resize a disk when the disk size does not meet your business requirements or the disk is full, see Expand disk volumes.
For more information about real-time disk usage, see Overview of container storage monitoring.