Community Blog Use a Local Disk Through LocalVolume Provisioner in a Kubernetes Cluster

Use a Local Disk Through LocalVolume Provisioner in a Kubernetes Cluster

In this article, we'll explore how to use and configure Alibaba Cloud local disks in a Kubernetes system using LocalVolume.

By Kan Junbao


Alibaba Cloud provides local disk configurations for some Elastic Compute Service ECS instances. Local disks have the advantages of low latency, high random IOPS, high throughput, and high cost effectiveness, and has great advantages in some applications with high performance requirements.

In the Kubernetes system, local disks can be used through HostPath, LocalVolume, and other types of PV.

  • HostPath: The volume itself does not contain scheduling information. If you want to fix each pod on a node, you need to configure scheduling information, such as nodeSelector, for the pod;
  • LocalVolume: The volume itself contains scheduling information, and the pods using this volume will be fixed on a specific node, which can ensure data continuity.

This document describes how to use the Alibaba Cloud local disk using LocalVolume.

  • Use LocalVolume Provisioner to automatically create LocalVolume, and the PV comes with the information about the node to which it belongs.
  • Create a PVC to consume LocalVolume. Pods using this PVC will be scheduled to the node specified by the PV.

How Provisioner Works

1) Introduction to Local Disks:

To learn more about ECS local disks, refer to Local disks

Local disks cannot be mounted or unmounted;

Local disks and ECS must coexist. The number of local disks is determined when ECS is created;

The type of the machine determines the number of local disks, which can be obtained through API;

By default, local disks are mounted from /dev/vdb, which may be different from custom images.

2) Mounting Principle:


Provisioner mounts local disks to the target directory at startup:

The number of local disks in this machine can be queried through API;

Local disks are mounted from the initiating device (/dev/vdb, by default) to the target directory in turn (the initiating device can be configured);

The device detects the file system. If the device does not format the file system, a file system is created;

Provisioner polls and checks the target directory in real time, and creates a PV based on the target directory list:

When a PV is created, StorageClass can support the configuration of reclaimPolicy and volumeBindingMode;

After LocalVolume PV is deleted, it is automatically created again;

PV name: local-pv-{node name}-{device Name}. The PV name will be added to the label;

Deploy Alibaba LocalVolume Provisioner

1) Create ConfigMap:

The configuration parameters of Provisioner are placed in the ConfigMap, provisioner-config:

vendor: optional. If it is alibabacloud, the Alibaba Cloud local disk is enabled;

hostDir: the mount directory of the local disk on the host;

mountDir: the mount directory of the local disk on the container after it is mounted to the host;

volumeMode: the mount mode, supporting file systems and blocks;

fsType: the file system type, which defaults to ext4;

deviceStartWith: optional. It defaults to vdb, the starting name of the local disk;

mkFSOptions: optional. It is null by default, indicating that the parameter after "mkfs. *" is performed;

mountOptions: optional. It is null by default, indicating that the parameter of "mount -o" is performed;

apiVersion: v1
kind: ConfigMap
  name: local-provisioner-config
  namespace: kube-system
  storageClassMap: |
       vendor: alibabacloud
       hostDir: /mnt/disks
       mountDir: /mnt/disks
         - "/scripts/shred.sh"
         - "2"
       volumeMode: Filesystem
       fsType: ext4
       deviceStartWith: vdb
       mkFSOptions: ""
       mountOptions: "nodelalloc"

2) Create StorageClass

LocalVolume StorageClass does not actually go to Provisioner PV, but is used to define configuration details, such as reclaimPolicy and volumeBindingMode.

reclaimPolicy: It supports Retain and Delete;

volumeBindingMode: It supports Immediate and WaitForFirstConsumer.

kind: StorageClass
apiVersion: storage.k8s.io/v1
  name: local-volume
provisioner: kubernetes.io/no-provisioner
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer

3) Create a Provisioner Service

For deployment through Daemonset, set hostPID and hostNetwork to true;

The privileged permission is added;

Deployment through AK or STS is supported;

The AK-free STS function is supported;

The node needs to configure the RAM role information, and the role has access to the cloud disk.

When using the STS function, you do not need to configure ACCESS_KEY_ID and ACCESS_KEY_SECRET parameters;

To configure STS, see Create custom authorization policies

apiVersion: extensions/v1beta1
kind: DaemonSet
  name: local-volume-provisioner
  namespace: kube-system
    app: local-volume-provisioner
      app: local-volume-provisioner
        app: local-volume-provisioner
      hostPID: true
      hostNetwork: true
      serviceAccountName: admin
        - image: registry.cn-hangzhou.aliyuncs.com/plugins/local-volume-provisioner:v1.12-7802d35-aliyun
          imagePullPolicy: "Always"
          name: provisioner
            privileged: true
          - name: MY_NODE_NAME
                fieldPath: spec.nodeName
          - name: ACCESS_KEY_ID
            value: ""
          - name: ACCESS_KEY_SECRET
            value: ""
            - mountPath: /etc/provisioner/config
              name: provisioner-config
              readOnly: true
            - mountPath:  /mnt/disks
              name: local
              mountPropagation: "HostToContainer"
            - mountPath: /etc/kubernetes
              name: etc
        - name: provisioner-config
            name: local-provisioner-config
        - name: local
            path: /mnt/disks
        - name: etc
            path: /etc/kubernetes

Consume LocalVolume

apiVersion: v1
kind: PersistentVolumeClaim
  name: myclaim
  - ReadWriteOnce
      storage: 10Gi
  storageClassName: local-volume
kind: Pod
apiVersion: v1
  name: mypod
    - name: myfrontend
      image: nginx
      - mountPath: "/data"
        name: mypd
    - name: mypd
        claimName: myclaim
0 0 0
Share on

Alibaba Container Service

114 posts | 26 followers

You may also like


Alibaba Container Service

114 posts | 26 followers

Related Products