In a Sandboxed-Container runtime, the root file systems of all containers on a node share one disk. Some application containers may occupy all I/O resources of the disk, which affects other containers on the node. This topic provides examples to describe how to set block I/O limits on the root file systems of containers in a pod to avoid I/O resource contention among containers.

Background information

Block I/O limits on a root file system are listed as follows:
  • read_bps: the maximum number of bytes that can be read per second
  • write_bps: the maximum number of bytes that can be written per second
  • read_iops: the maximum read operations per second
  • write_iops: the maximum write operations per second

Parameters

When you set block I/O limits on the root file systems of containers in a pod, add a key-value pair in annotations of the pod. Set the following parameters:
  • In annotations, set key to io.kubernetes.container.blkio.
  • In annotations, set value in the following format:
    '{
      "device_read_bps": [
        {
          "device": "rootfs",
          "value": "${VALUE}"
        }
      ],
      "device_write_bps": [
        {
          "device": "rootfs",
          "value": "${VALUE}"
        }
      ],
      "device_read_iops": [
        {
          "device": "rootfs",
          "value": "${VALUE}"
        }
      ],
      "device_write_iops": [
        {
          "device": "rootfs",
          "value": "${VALUE}"
        }
      ]
    }'
    Parameter Type Description
    device_read_bps Array The maximum number of bytes that can be read from the device per second.
    device_read_bps[].device String The absolute path of the device, for example, /dev/sda1. The only valid value is rootfs.
    device_read_bps[].value String The limit value. The unit can be k, m, or g. For example, 20m indicates that a maximum of 20 MB data can be read per second.
    device_write_bps Array The maximum number of bytes that can be written to the device per second.
    device_write_bps[].device String The absolute path of the device, for example, /dev/sda1. The only valid value is rootfs.
    device_write_bps[].value String The limit value. The unit can be k, m, or g. For example, 20m indicates that a maximum of 20 MB data can be written per second.
    device_read_iops Array The maximum number of read operations per second on the device.
    device_read_iops[].device String The absolute path of the device, for example, /dev/sda1. The only valid value is rootfs.
    device_read_iops[].value String The limit value. For example, 200 indicates that the maixmum number of read operations per second is 200.
    device_write_iops Array The maximum number of write operations per second on the device.
    device_write_iops[].device String The absolute path of the device, for example, /dev/sda1. The only valid value is rootfs.
    device_write_iops[].value String The limit value. For example, 200 indicates that the maximum number of write operations per second is 200.

Example

The following example describes how to set block I/O limits on the root file systems of containers in a pod by deploying a test application based on Deployment. In the example, read_bps is set to 20m, read_iops is set to 200, and write_iops is set to 300. Set parameters based on the following template:
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: iops-app
  name: iops-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: iops-app
  template:
    metadata:
      labels:
        app: iops-app
      annotations:
        io.kubernetes.container.blkio: '{"device_read_bps":[{"device":"rootfs","value":"20m"}],"device_write_bps":[{"device":"rootfs","value":"20m"}],"device_read_iops":[{"device":"rootfs","value":"200"}],"device_write_iops":[{"device":"rootfs","value":"300"}]}'
    spec:
      runtimeClassName: runv
      nodeSelector:
        alibabacloud.com/container-runtime: Sandboxed-Container.runv
      containers:
        - image: docker.io/centos:7
          command: ["/bin/sh","-c"]
          args: ["while(true);do /bin/sleep 10;done"]
          name: iops-app
          resources:
            requests:
              memory: "1Gi"
              cpu: 1
            limits:
              memory: "1Gi"
              cpu: 1
          volumeMounts:
            - name: host-time
              mountPath: /etc/localtime
      volumes:
        - name: host-time
          hostPath:
            path: /etc/localtime

Testing and verification

  1. Run the following commands to log on to a pod that holds sandboxed containers.
    kubectl get pod
    kubectl exec -ti ${POD} /bin/sh
  2. Run the following commands to perform a stress test on sandboxed containers:
    # Run the following commands on the pod:
    cd /
    dd if=/dev/zero of=test oflag=direct bs=1M count=10
    dd if=/dev/zero of=test oflag=direct bs=2M count=20

Result

Result

As shown in the preceding figure, the data write speed is about 20 MB/s, which does not exceed the threshold in annotations.