All Products
Search
Document Center

Container Service for Kubernetes:Expand a disk volume without service interruptions

Last Updated:Sep 07, 2023

In Kubernetes 1.16, the feature of expanding disk volumes without service interruptions is in public preview. Container Service for Kubernetes (ACK) allows you to use the Container Storage Interface (CSI) plug-in to expand a disk volume without service interruptions for clusters that run Kubernetes 1.16 and later. This topic describes how to expand a disk volume without service interruptions.

Usage notes

  • Limits

    To ensure the security of volume expansion, when the feature of automatically backing up data and expanding disk volumes without service interruptions is disabled, you can expand only disk volumes of 2,000 GiB or smaller without service interruptions. After you enable this feature, you can expand disk volumes of 32 TiB or smaller without service interruptions.

  • Data backup

    Before you expand a disk volume, create a snapshot for the disk in case data issues occur during disk volume expansion.

  • Scenarios

    • You can expand only dynamically provisioned persistent volumes (PVs) without service interruptions, which are mounted by using persistent volume claims (PVCs) that contain the StorageClassName parameter.

    • You cannot expand disk volumes of the inline type. This type of disk volume is not created by using PVs and PVCs.

    • You cannot expand volumes that are associated with basic disks without service interruptions.

    • Specify AllowVolumeExpansion: True for the StorageClass. The AllowVolumeExpansion parameter is automatically set to True for StorageClasses that are created by ACK. For StorageClasses that are manually created, you must manually set the AllowVolumeExpansion parameter to True.

    • Pods must remain in the Running state during volume expansion.

  • Plug-in version

    Make sure that the CSI plug-in is updated to the latest version.

Grant the ResizeDisk permission to the Resource Access Management (RAM) role of the cluster

Before you expand a disk volume without service interruptions, you must grant the ResizeDisk permission to the RAM role of the cluster. Perform the following steps to grant the ResizeDisk permission based on the cluster type and the volume plug-in that is used:

CSI plug-in is updated to the latest version

  1. Log on to the ACK console.

  2. In the left-side navigation pane, click Clusters.

  3. On the Clusters page, click Details in the Actions column of the cluster that you want to manage.

  4. In the left-side navigation pane, click Cluster Information.

  5. Click the Cluster Resources tab and click the hyperlink next to Master RAM Role.

  6. In the RAM console, grant the ResizeDisk permission to the RAM role. For more information about how to grant the permissions, see Modify the document and description of a custom policy.resizedisk

ACK dedicated clusters that use FlexVolume or ACK managed clusters

Perform the preceding Step 1 to Step 4 and click the hyperlink next to Master RAM Role.

Expand a disk volume without starting the application pods

  1. Use a command-line tool to connect to the cluster. For more information, see Step 2: Select a type of cluster credentials.

    In this example, the pod that you want to manage is in the following state.

    Run the following command to query pod information:

    kubectl get pod

    Expected output:

    <your-pod-name>         1/1     Running   0          42s

    Run the following command to query the volume that is mounted to the pod:

    kubectl exec <your-pod-name>  -- df /data

    Expected output:

    Filesystem     1K-blocks  Used   Available Use% Mounted on
    /dev/vdb       20511312   45080  20449848  1%   /data

    Run the following command to query PVC information:

    kubectl get pvc

    Expected output:

    NAME              STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS                     AGE
    <your-PVC-name>   Bound    d-wz9hpoifm43yn9zi****   20Gi       RWO            alicloud-disk-topology-alltype   57s

    Run the following command to query PV information:

    kubectl get pv

    Expected output:

    NAME                     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS     CLAIM                     STORAGECLASS                     REASON   AGE
    d-wz9hpoifm43yn9zi****   20Gi       RWO            Delete           Bound      default/<your-PVC-name>   alicloud-disk-topology-alltype            65s
  2. Run the following command to expand the disk volume when the requirements in the Usage notes section are met:

    kubectl patch pvc <your-PVC-name> -p '{"spec":{"resources":{"requests":{"storage":"30Gi"}}}}'

    Wait 1 minute and then check whether the PV is expanded.

    Run the following command to query PV information:

    kubectl get pv d-wz9hpoifm43yn9zi****

    Expected output:

    NAME                     CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                    STORAGECLASS                     REASON   AGE
    d-wz9hpoifm43yn9zi****   30Gi       RWO            Delete           Bound    default/<your-PVC-name>   alicloud-disk-topology-alltype           5m23s

    Run the following command to query PVC information:

    kubectl get pvc

    Expected output:

    NAME             STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS                     AGE
    <your-PVC-name>   Bound    d-wz9hpoifm43yn9zi****   30Gi       RWO           alicloud-disk-topology-alltype   5m10s

    Run the following command to query the volume that is mounted to the pod:

    kubectl exec <your-pod-name> -- df /data

    Expected output:

    Filesystem     1K-blocks  Used   Available  Use%  Mounted on
    /dev/vdb       30832548   45036  30771128   1%    /data

    To expand a disk volume without service interruptions, you need to only run the preceding command.

Configure automatic backup for ESSD expansion without service interruptions

The following figure shows the procedure for automatically backing up and expanding an Enhanced SSD (ESSD) without service interruptions.

essd

Sequence number

Description

Create a request for expanding an ESSD without service interruptions.

(2)

Create an instant access (IA) snapshot for the ESSD.

(3)

Expand the ESSD.

(4)

Expand the file system of the ESSD.

(5)

Delete the IA snapshot after the ESSD is successfully expanded.

Note

To configure the system to automatically back up an ESSD before it expands the ESSD without service interruptions, perform the following steps:

  1. To enable automatic backup, set the volumeExpandAutoSnapshot parameter to forced when you create the StorageClass.

  2. Run the following command to query the StroageClass information:

    kubectl describe sc <your-storageClass-name> | grep volumeExpandAutoSnapshot

    Expected Annotations output:

    "volumeExpandAutoSnapshot":"forced"

    Expected Parameters output:

    volumeExpandAutoSnapshot=forced
  3. Run the following command to query information about the PVC:

    kubectl get pvc

    Expected output:

    NAME              STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS               AGE
    <your-PVC-name>   Bound    d-2ze39o7sv976dz1y****   20Gi       RWO            <your-storageClass-name>   13h
  4. If all requirements other than the data backup requirements in Usage notes for snapshots and Usage notes for disk expansion are met and the disk is an ESSD, you can perform the following steps to back up and expand the disk:

    1. Run the following command to back up and expand the ESSD:

      kubectl patch pvc <your-PVC-name> -p '{"spec":{"resources":{"requests":{"storage":"30Gi"}}}}'

      Wait until the expansion is complete.

    2. Run the following command to query information about the PVC:

      kubectl describe pvc <your-PVC-name>

      Expected Events output:

        Normal   SnapshotCreatedSuccessfully  91s                csi-controller-server                                                                                  ControllerExpandVolume:: Snapshot create successful: snapshotName[volume-expand-auto-snapshot-d-2ze39o7sv976dz1y****-2022-08-17-21:30:48], sourceId[d-2ze39o7sv976dz1y****], snapshotId[s-2ze85t4rk5d695ha****]

    Snapshots are automatically created to back up disks that need to be expanded.

    • If a disk is successfully expanded, the snapshot of the disk is automatically deleted.

    • If a disk fails to be expanded, you can use the snapshot of the disk to manually restore data. The default retention period of the snapshot is 24 hours.