Container Service for Kubernetes:Enable persistent storage for a StatefulSet

Scenarios

StatefulSets provide the following characteristics:

• Predefined deployment order: The pods provisioned by a StatefulSet are deployed or scaled out from 0 to N-1 in sequence. The system must wait until all of the preceding pods reach the Running or Ready state before it can deploy another pod.

• Predefined scale-in order: The pods provisioned by a StatefulSet are scaled in or deleted from N-1 to 0 in sequence. The system must wait until all of the preceding pods are deleted before it can delete another pod.

• Consistent network identifiers: After a pod provisioned by a StatefulSet is rescheduled, its PodName and HostName values remain unchanged.

• Stable data persistence: After a pod provisioned by a StatefulSet is rescheduled, the pod can still access the same persisted data.

When you add the VolumeClaimTemplates parameter to a StatefulSet to mount disk volumes to the pods provisioned by the StatefulSet, the following features are achieved:

• When you create a StatefulSet, you can use the VolumeClaimTemplate parameter to automatically create PVCs and PVs. A pod and the PVC created for the pod have the same serial number. For example, the PVC created for the web-0 pod is named disk-essd-web-0 and the PVC created for the web-1 pod is named disk-essd-web-1.

• When a new pod is created during the scale-out of a StatefulSet, if a PVC and a PV have the same serial number as the pod, the pod uses the PVC and PV. If not, the system creates a new PVC and a new PV that have the same serial number as the pod.

• When a pod is deleted during the scale-out of a StatefulSet, the PVC and PV used by the pod are retained.

• When you delete a pod provisioned by a StatefulSet, the PVC and PV used for the pod are retained. In this case, if the pod is recreated, the previously used PVC and PV are used by the pod.

• You can enable persistent storage for a StatefulSet by mounting disks to the StatefulSet as PVs.

Create a StatefulSet and mount a disk volume to the StatefulSet.

1. Create a file named statefulset.yaml and add the following YAML content to the file:

   Use the following YAML template to create a StatefulSet and a Service. The StatefulSet provisions two pods and uses the volumeClaimTemplates parameter to create PVCs and PVs. In this case, a performance level 1 (PL1) Enterprise SSD (ESSD) that has a capacity of 20 GiB is mounted to each pod.

   apiVersion: v1
   kind: Service
   metadata:
     name: nginx
     labels:
       app: nginx
   spec:
     ports:
     - port: 80
       name: web
     clusterIP: None
     selector:
       app: nginx
   ---
   apiVersion: apps/v1
   kind: StatefulSet
   metadata:
     name: web
   spec:
     selector:
       matchLabels:
         app: nginx
     serviceName: "nginx"
     replicas: 2
     template:
       metadata:
         labels:
           app: nginx
       spec:
         containers:
         - name: nginx
           image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
           ports:
           - containerPort: 80
             name: web
           volumeMounts:
           - name: disk-essd
             mountPath: /data
     volumeClaimTemplates:
     - metadata:
         name: disk-essd
       spec:
         accessModes: [ "ReadWriteOnce" ]
         storageClassName: "alicloud-disk-essd"
         resources:
           requests:
             storage: 20Gi

   The following table describes the parameters in the volumeClaimTemplates parameter.

   Parameter	Description
   accessModes	The access mode of the volume.
   storageClassName	The name of the StorageClass that you want to use. In this example, the alicloud-disk-essd StorageClass, which is a default StorageClass provided by Container Service for Kubernetes (ACK), is used. This StorageClass is used to provision a disk volume based on a PL1 ESSD. Note You are charged for the disks you use on a pay-as-you-go basis. For more information, see Billing of block storage devices and Prices of block storage devices.
   storage	The capacity of the disk volume.

2. Create a StatefulSet.

   kubectl create -f statefulset.yaml

3. Check whether the pods provisioned by the StatefulSet are deployed.

   kubectl get pod -l app=nginx

   Expected output:

   NAME    READY   STATUS    RESTARTS   AGE
   web-0   1/1     Running   0          70s
   web-1   1/1     Running   0          55s

4. Query PVCs in the cluster.

   kubectl get pvc

   The following output shows that PVCs and PVs are created for the pods:

   NAME              STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS         VOLUMEATTRIBUTESCLASS   AGE
   disk-essd-web-0   Bound    d-uf6727q3zpydr645****   20Gi       RWO            alicloud-disk-essd   <unset>                 93s
   disk-essd-web-1   Bound    d-uf63en5prlz6halx****   20Gi       RWO            alicloud-disk-essd   <unset>                 78s

Verify the changes of PVCs when the StatefulSet is scaled

Check whether the PVCs used by the StatefulSet are retained when you delete a pod of the StatefulSet

1. Query the PVCs used by the pods of the StatefulSet.

   In the following example, the web-1 pod is used.

   kubectl describe pod web-1 | grep ClaimName

   Expected output:

   ClaimName:  disk-essd-web-1

2. Delete a pod of the StatefulSet.

   In the following example, the web-1 pod is deleted:

   kubectl delete pod web-1

3. Query the pods created for the StatefulSet.

   kubectl get pod -l app=nginx

   The following output shows that a new pod is created. The new pod has the same name as the previously deleted pod.

   NAME    READY   STATUS    RESTARTS   AGE
   web-0   1/1     Running   0          7m8s
   web-1   1/1     Running   0          11s
   web-2   1/1     Running   0          2m27s

4. Query PVCs in the cluster.

   kubectl get pvc

   The following output shows that the new pod uses the PVC used by the previously deleted pod:

   NAME              STATUS   VOLUME                   CAPACITY   ACCESS MODES   STORAGECLASS         VOLUMEATTRIBUTESCLASS   AGE
   disk-essd-web-0   Bound    d-uf6727q3zpydr645****   20Gi       RWO            alicloud-disk-essd   <unset>                 7m55s
   disk-essd-web-1   Bound    d-uf63en5prlz6halx****   20Gi       RWO            alicloud-disk-essd   <unset>                 7m40s
   disk-essd-web-2   Bound    d-uf6f3nbkzljbqyb0****   20Gi       RWO            alicloud-disk-essd   <unset>                 5m38s

Check whether the disk volume can be used to persist data

1. Query the mount path of the disk volume.

   In the following example, the web-1 pod is used:

   kubectl exec web-1 -- ls /data

   Expected output:

   lost+found

2. Create a file in the mount path of the disk volume.

   In the following example, the web-1 pod is used:

   kubectl exec web-1 -- touch /data/test
   kubectl exec web-1 -- ls /data

   Expected output:

   lost+found
   statefulset

3. Delete a pod.

   In the following example, the web-1 pod is deleted:

   kubectl delete pod web-1

4. Query the newly created pod.

   kubectl get pod -l app=nginx

   Expected output:

   NAME    READY   STATUS    RESTARTS   AGE
   web-0   1/1     Running   0          10m
   web-1   1/1     Running   0          10s
   web-2   1/1     Running   0          6m14s

5. Check whether the original disk is mounted to the newly created pod and whether the file still exists on the disk.

   kubectl exec web-1 -- ls /data

   The following output shows that the test file still exists on the disk. This indicates that the disk can be used to persist data.

   lost+found
   statefulset