Expand disk volumes manually when PVC update is not supported - Container Service for Kubernetes

Use this procedure when your cluster runs Kubernetes earlier than 1.16, or when the disk is a basic disk. In these cases, you cannot expand a disk volume by editing the persistent volume claim (PVC). Instead, resize the disk on the Elastic Compute Service (ECS) side and extend the file system manually.

Important

This method does not modify the PVC or persistent volume (PV). After resizing, the capacity values displayed in the PVC and PV remain unchanged even though the actual disk is larger. For clusters that support it, use online disk volume expansion instead, which keeps the PVC and PV in sync automatically.

The examples in this topic use an application backed by a dynamically provisioned disk volume.

Choose a resizing method

Method	Downtime required	When to use
Online resizing	No	You cannot afford application downtime. Note that if disk I/O throughput is high during file system expansion, an I/O error may occur.
Offline resizing	Yes	Data integrity is the priority, or the disk is in Pending state after the application pauses.

Important

Basic disks cannot be resized without downtime. Use offline resizing for basic disks.

Online resizing

Prerequisites

Before you begin, make sure that you have:

Access to the cluster via kubectl. For setup instructions, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster or Manage Kubernetes clusters via kubectl in Cloud Shell
A running pod with a disk volume attached
A disk that is not a basic disk

Step 1: Record the current state

Before resizing, record the current disk capacity so you can verify the change later.

Confirm the pod is in the Running state.

kubectl get pod

Expected output:

NAME          READY   STATUS    RESTARTS   AGE
disk-test-0   1/1     Running   0          38s

Get the disk ID and current capacity from the PVC.

kubectl get pvc

Expected output:

NAME       STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
disk-pvc   Bound    d-uf628m33r5rsbi******   20Gi       RWO            disk           <unset>                 64s

Note the current capacity (20 GiB) and the disk ID shown in the VOLUME column (d-uf628m33r5rsbi******).

Step 2: Resize the disk on the ECS side

Log on to the ECS console.
In the left-side navigation pane, choose Storage & Snapshots > Block Storage.
In the top navigation bar, select the region and resource group.
Find the disk by its ID, select it, and click Resize at the bottom of the page.

Complete the resize wizard:

Determine disk and read notes: Confirm the disk information and click I Understand Risk and Have Backed Up Data. Proceed.

Configure resizing method and new size: Set the parameters and click OK.

Parameter	Description	Example
New disk size	New capacity. The value cannot be less than the current disk size.	30 GiB
Resizing method	Select Online Resizing. The new size takes effect immediately without restarting the instance.	Online resizing

Confirm resizing results: Confirm the disk is resized and click Next Step, Extend Partitions and File Systems. > Note: After the disk is resized, extend the file system to make the new capacity available to the application.
Extend partitions and file systems: Select Use Cloud Assistant, check Authorize Cloud Assistant to Perform This Operation, and click Authorize Cloud Assistant. After Cloud Assistant completes, confirm that both the disk and file system are at the new size.

Step 3: Verify the expansion

Check the file system size inside the pod to confirm the expansion is complete.

kubectl exec -it disk-test-0 -- df /data

The output should show the updated capacity (30 GiB):

Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/vdb        30787536    24  30771128   1% /data

If the capacity still shows the old value, wait a few minutes and retry. The file system expansion may take a short time to propagate.

Offline resizing

Offline resizing pauses the application by scaling replicas to 0, resizes the disk, extends the file system manually, and then restarts the application.

Prerequisites

Before you begin, make sure that you have:

Access to the cluster via kubectl. For setup instructions, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster or Manage Kubernetes clusters via kubectl in Cloud Shell
A running pod with a disk volume attached
Access to an ECS instance you can attach the disk to temporarily

Step 1: Record the disk ID and pause the application

Confirm the pod is in the Running state.

kubectl get pod

Expected output:

NAME          READY   STATUS    RESTARTS   AGE
disk-test-0   1/1     Running   0          38s

Get the disk ID from the PVC.

kubectl get pvc

Expected output:

NAME       STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
disk-pvc   Bound    d-uf628m33r5rsbi******   20Gi       RWO            disk           <unset>                 64s

Scale the StatefulSet to 0 replicas to pause the application and release the disk.
```
kubectl scale sts disk-test --replicas=0
```
Expected output:
```
statefulset.apps/disk-test scaled
```

Step 2: Resize the disk on the ECS side

Log on to the ECS console.
In the left-side navigation pane, choose Storage & Snapshots > Block Storage.
In the top navigation bar, select the region and resource group.
Find the disk by its ID, select it, and click Resize at the bottom of the page.

Complete the resize wizard:

Determine disk and read notes: Confirm the disk information and click I Understand Risk and Have Backed Up Data. Proceed.

Configure resizing method and new size: Set the parameters and click OK.

Parameter	Description	Example
New disk size	New capacity. The value cannot be less than the current disk size.	30 GiB
Resizing method	Because the application is paused, the disk is in Pending state. In this case, you can select only Offline Resizing.	Offline resizing

Confirm resizing results: Confirm the disk is resized and click Next Step, Extend Partitions and File Systems.
Extend partitions and file systems: Select Use Documentation and follow the steps in the Extend Partitions and File Systems (Linux) section. > Note: Use Cloud Assistant is unavailable when the disk is in Pending state. Extend the file system manually in the next step.

Step 3: Extend the file system

The following steps apply to unpartitioned disks.

Important

Do not use partitioned disks in Kubernetes.

If an unpartitioned disk is mounted as a PV, creating partitions on it can damage the file system and cause data loss.
If a partitioned disk is mounted as a PV, expand the partitioned disk first, then extend the file system. See Extend the partitions and file systems of disks on a Linux instance or Extend the partitions and file systems of disks on Windows instances.

Attach the disk to an ECS instance.
1. On the Block Storage page of the ECS console, find the disk and click MoreAttach in the Actions column.
2. On the Attach Cloud Disk page, select the target ECS instance and click Next. > Important: Do not select any disk-related release behavior.
3. On the Partition Disk and Create and Mount File Systems page, click Configure Later in the Initialization Method section.
4. Confirm that the disk status changes to In Use.
Connect to the ECS instance. For instructions, see Methods for connecting to an ECS instance.
Get the device identifier of the disk. Use the disk serial number to identify the device. The disk ID maps to the serial number as: disk ID = d-{serial number}. For more information, see Query the serial numbers of block storage devices. Replace <serial> with the actual serial number:
Important
Disks created before June 10, 2020 do not support the serial number feature. To identify the device, run ls /dev/vd* before and after attaching the disk, then compare the output to find the new device.
```
realpath /dev/disk/by-id/virtio-<serial>
```
Expected output (the device identifier):
```
/dev/vdb
```

Extend the file system. Replace /dev/vdb with your actual device identifier.

resize2fs /dev/vdb

If the command returns the following error, run e2fsck first, then retry resize2fs:

resize2fs 1.46.0 (29-Jan-2020)
Please run 'e2fsck -f /dev/vdb' first.

e2fsck -f /dev/vdb
resize2fs /dev/vdb

A successful resize produces output similar to:

resize2fs 1.46.0 (29-Jan-2020)
Resizing the filesystem on /dev/vdb to 7864320 (4k) blocks.
The filesystem on /dev/vdb is now 7864320 (4k) blocks long.

Verify the file system size.

Mount the disk to a temporary folder.

mkdir /mnt/disk
mount /dev/vdb /mnt/disk/

Check the capacity.

df /mnt/disk/

The output confirms the disk is now 30 GiB.

Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/vdb        30787536    24  30771128   1% /mnt/disk

Unmount the disk.
```
umount /mnt/disk
```

Detach the disk from the ECS instance. For instructions, see Detach a data disk.

Step 4: Restart the application

Scale the StatefulSet back to one replica.

kubectl scale sts disk-test --replicas=1

Expected output:

statefulset.apps/disk-test scaled

Confirm the pod is running.

kubectl get pod

Expected output:

NAME          READY   STATUS    RESTARTS   AGE
disk-test-0   1/1     Running   0          5s

Verify the file system inside the pod reflects the new size.
```
kubectl exec -it disk-test-0 -- df /data
```
Expected output:
```
Filesystem     1K-blocks  Used Available Use% Mounted on
/dev/vdb        30787536    24  30771128   1% /data
```
The mount path /data now shows 30 GiB, confirming the expansion is complete.

What's next

To keep PVC and PV capacity in sync with the actual disk size, follow the steps in Expand a disk volume without service interruptions.