Expand disk volumes in ACK clusters - Container Service for Kubernetes

If the Kubernetes version of your cluster is earlier than 1.16 or the disk is a basic disk, you cannot perform online disk volume expansion by updating the persistent volume claim (PVC) that is used to mount the disk volume. In this case, you must manually resize the disk and extend the file system.

Applicable scope

This topic describes how to manually resize the disk without modifying the PVC. If the Kubernetes version of your cluster is earlier than 1.16 or you cannot perform online disk volume expansion, you can use this method.

Important

In this mode, you must resize the disk on the Elastic Compute Service (ECS) side. The resources in the cluster are not affected, and the capacity of the PVC and PV on the cluster side remains unchanged. To ensure that the disk capacity values displayed in the PV and PVC are the same as the actual disk capacity, we recommend that you perform the steps in the Expand a disk volume without service interruptions topic.

You can select online or offline resizing based on your business requirements.

Online resizing: If the I/O throughput of a disk is high when you expand the file system of the disk, an I/O error may occur in the file system. If you select this method, you do not need to restart the application.
Offline resizing: After you pause the application, the disk I/O is suspended to ensure datas security. However, the application is suspended for a specific period of time.

Important

Basic disks cannot be resized without service interruption.

The following example describes how to expand a disk volume by using an application that is created in a dynamically provisioned disk volume.

Online resizing

Resize a disk without service interruptions

Step 1: View disk information

Connect to the cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster or Manage Kubernetes clusters via kubectl in Cloud Shell.
View information about the pod to which the disk is attached and make sure that the pod is in the Running state.
```
kubectl get pod 
```
Expected output:
```
NAME          READY   STATUS    RESTARTS   AGE
disk-test-0   1/1     Running   0          38s
```

View the PVC to obtain the disk ID.

kubectl get pvc

The following command output indicates that the PV bound to the PVC is d-uf628m33r5rsbi****** (the ID of the cloud disk) and the capacity is 20 GiB.

NAME       STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
disk-pvc   Bound    d-uf628m33r5rsbi******   20Gi       RWO            disk           <unset>                 64s

Step 2: Resize the disk and file system

Log on to the ECS console.
In the left-side navigation pane, choose Block Storage.
In the top navigation bar, select the region and resource group of the resource that you want to manage.
Find the disk attached to the application is attached by using the disk ID, select the disk, and then click Resize in the lower part of the page.

On the wizard page that appears, follow the instructions to complete the resizing operation.

In the Determine Disk and Read Notes step, confirm the disk information, read the notes, and click I Understand Risk and Have Backed Up Data. Proceed.

In the Configure Resizing Method and New Size step, configure the parameters and click OK.

Parameter	Description	Example
New Disk Size	Specify the capacity of the disk. The value cannot be less than the current disk size.	30 GiB
Resizing Method	Click Online Resizing. The new disk size immediately takes effect without the need to restart the instance.	Online resizing

In the Confirm Resizing Results step, confirm that the disk is resized and click Next Step, Extend Partitions and File Systems.
Note
After the disks are resized, you must expand the file systems to update the storage capacity of the application.
In the Extend Partitions and File Systems step, select Use Cloud Assistant, select Authorize Cloud Assistant to Perform This Operation, and then click Authorize Cloud Assistant.
After you perform the operations, make sure that the disk and file system are resized.

Offline resizing

To pause the application, you can set the number of replica to 0. After the disk is resized, restart the application.

Step 1: View the disk information and pause the application

Connect to the cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster or Manage Kubernetes clusters via kubectl in Cloud Shell.
View information about the pod to which the disk is attached and make sure that the pod is in the Running state.
```
kubectl get pod 
```
Expected output:
```
NAME          READY   STATUS    RESTARTS   AGE
disk-test-0   1/1     Running   0          38s
```

View the PVC to obtain the disk ID.

kubectl get pvc

The following command output indicates that the PV bound to the PVC is d-uf628m33r5rsbi****** (the ID of the cloud disk) and the capacity is 20 GiB.

NAME       STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
disk-pvc   Bound    d-uf628m33r5rsbi******   20Gi       RWO            disk           <unset>                 64s

Scale in the replicas of the application to 0 and pause the application.
```
kubectl scale sts disk-test --replicas=0
```
Expected output:
```
statefulset.apps/disk-test scaled
```

Step 2: Resize the disk

Log on to the ECS console.
In the left-side navigation pane, choose Block Storage.
In the top navigation bar, select the region and resource group of the resource that you want to manage.
Find the disk attached to the application by using the disk ID, select the disk, and then click Resize in the lower part of the page.

On the wizard page that appears, follow the instructions to complete the resizing operation.

In the Determine Disk and Read Notes step, confirm the disk information, read the notes, and click I Understand Risk and Have Backed Up Data. Proceed.

In the Configure Resizing Method and New Size step, configure the parameters and click OK.

Parameter	Description	Example
New Disk Size	Specify the capacity of the disk. The value cannot be less than the current disk size.	30 GiB
Resizing Method	After the application is paused, the disk is in the Pending state. In this case, you can select only Offline Resizing.	Offline resizing

In the Confirm Resizing Results step, confirm that the disk is resized and click Next Step, Extend Partitions and File Systems.
In the Extend Partitions and File Systems step, select Use Documentation and view the files in the Extend Partitions and File Systems (Linux) section.
Note
- After the disks are resized, you must expand the file systems to update the storage capacity of the application.
- The Use Cloud Assistant option cannot be selected because the disk is in the Pending state. You can manually resize the file system.

Step 3: Extend the file system

The following operations are performed for unpartitioned disks.

Important

We recommend that you do not use partitioned disks in Kubernetes.

If an unpartitioned disk is mounted as a PV, you cannot manually create partitions for the disk. This may damage the file system and cause data loss.
If a partitioned disk is mounted as a PV, expand the file system after you expand the partitioned disk. For more information, see Extend the partitions and file systems of disks on a Linux instance or Extend the partitions and file systems of disks on Windows instances.

Attach the disk to an ECS instance.
1. On the Block Storage page of the ECS console, find the disk that you want to mount and click Attach in the Actions column.
2. On the Attach Cloud Disk page, select the ECS instance that you want to attach and click Next.
  Important
  Do not select the disk-related release behavior.
3. On the Partition Disk and Create and Mount File Systems page, click Configure Later in the Initialization Method section.
4. Make sure that the status of the disk changes to In Use.
Connect to an ECS instance. For more information, see Methods for connecting to an ECS instance.
Obtain the device identifier of the disk.
The following sample command is used. <serial> indicates the serial number of the disk. Replace the serial number based on your business requirements.
You can use the disk ID to quickly confirm the disk serial number (disk ID = d - {disk serial number}). For more information, see Query the serial numbers of block storage devices.
Important
If a disk was created before June 10, 2020, the serial number feature is not supported. You cannot run the following command to obtain the device identifier of the disk: You can run the ls /dev/vd* command before and after a disk is attached to view the disk list. Then, compare the difference between the command outputs to check the device identifier of the target disk.
```
realpath /dev/disk/by-id/virtio-<serial>
```
The following sample output is returned, which indicates that the device identifier is /dev/vdb.
```
/dev/vdb
```
Resize the file system.
The following sample command is used. Replace the value with the actual disk device identifier.
```
resize2fs /dev/vdb
```
If the following error message is returned, run the e2fsck -f /dev/vdb command as prompted and then run the resize2fs /dev/vdb command to resize the file system.
```
resize2fs 1.46.0 (29-Jan-2020)
Please run 'e2fsck -f /dev/vdb' first.
```
After you resize a file system, the following command output is displayed:
```
resize2fs 1.46.0 (29-Jan-2020)
Resizing the filesystem on /dev/vdb to 7864320 (4k) blocks.
The filesystem on /dev/vdb is now 7864320 (4k) blocks long.
```
Verify that the file system is resized.
1. Run the following command to create a temporary folder named /mnt/disk/ and mount the disk to the folder:
```
mkdir /mnt/disk
mount /dev/vdb /mnt/disk/
```
2. Check the size of the file system.
```
df /mnt/disk/
```
  The command output indicates that the capacity of /dev/vdb is expanded to 30 GiB.
```
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/vdb        30787536    24  30771128   1% /mnt/disk
```
3. Unmount the disk from the temporary folder.
```
umount /mnt/disk
```
Detach a disk from an ECS instance. For more information, see Detach a data disk.

Step 4: Restart the application

Resize the application to one replica and restart the application.
```
kubectl scale sts disk-test --replicas=1
```
Expected output:
```
statefulset.apps/disk-test scaled
```

Check whether the pod is restored.

kubectl get pod

Expected output:

NAME          READY   STATUS    RESTARTS   AGE
disk-test-0   1/1     Running   0          5s

Check the size of the file system.

kubectl exec -it disk-test-0 -- df /data

The command output indicates that the file system corresponding to the mount path is resized.

Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/vdb        30787536    24  30771128   1% /data