Enable resource isolation based on the L3 cache and MBA to ensure service quality for high-priority tasks - Container Service for Kubernetes

Cap L3 cache and memory bandwidth for BestEffort pods to protect latency-sensitive workloads on shared nodes.

Note See Pod QoS classes and Assign memory resources to containers and pods.

How it works

Intel Resource Director Technology (RDT) partitions hardware resources per node. ack-koordinator reads the ack-slo-config ConfigMap and applies resctrl group settings based on each pod's QoS class. By default, BestEffort (BE) pods get 30% of the L3 cache and latency-sensitive (LS) pods get 100%. Memory bandwidth defaults to 100% for both classes.

Prerequisites

Ensure that you have:

An ACK cluster with at least one Elastic Compute Service (ECS) bare metal instance whose CPU supports RDT (supported models).
ack-koordinator 0.8.0 or later is installed.

Billing

ack-koordinator is free. Costs may apply in two areas:

Node resource consumption: ack-koordinator is non-managed and runs on worker nodes. Specify per-module resource requests at install time to control overhead.
Prometheus metrics: If you enable Prometheus metrics for ack-koordinator and use Managed Service for Prometheus, the metrics count as custom metrics and are billed accordingly. Charges depend on cluster size and the number of applications. Review Managed Service for Prometheus billing before enabling, and monitor usage with Query observable data and bills.

Step 1: Check and enable RDT on the node kernel

Check if the kernel has RDT enabled:

cat /proc/cmdline

Look for l3cat and mba in the output:

# Other fields omitted. This example shows only the RDT portion of the BOOT_IMAGE field.
BOOT_IMAGE=... rdt=cmt,l3cat,l3cdp,mba

If l3cat and mba appear, RDT is already enabled — skip to Step 2.

Add RDT options to the kernel boot parameters. In /etc/default/grub, append the following to GRUB_CMDLINE_LINUX:

Important
Separate the RDT options from existing settings with a space. Do not overwrite other kernel parameters.
```
# Other fields omitted. This example shows only the RDT portion of the GRUB_CMDLINE_LINUX field.
GRUB_CMDLINE_LINUX="... rdt=cmt,mbmtotal,mbmlocal,l3cat,l3cdp,mba"
```

Regenerate the GRUB configuration:

# The file path may differ depending on your OS distribution.
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Restart the node:
```
sudo systemctl reboot
```
After the node restarts, rerun cat /proc/cmdline and verify that l3cat and mba appear.

Step 2: Configure L3 cache and MBA isolation

L3 cache and MBA isolation is controlled by the ack-slo-config ConfigMap in the kube-system namespace. Apply the ConfigMap, then label pods with their QoS class to activate the policy.

Apply the ConfigMap

Create configmap.yaml with the following content:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ack-slo-config
  namespace: kube-system
data:
  # resource-qos-config: configures QoS-based resource isolation features.
  # resctrlQOS: controls Intel RDT (L3 cache and MBA) isolation per QoS class.
  #   enable: set to true to activate L3 cache and MBA isolation for pods of this class.
  resource-qos-config: |
    {
      "clusterStrategy": {
        "beClass": {
          "resctrlQOS": {
            "enable": true
          }
        }
      }
    }

Then apply the ConfigMap:

If ack-slo-config already exists in kube-system, patch it to preserve other settings:
```
kubectl patch cm -n kube-system ack-slo-config --patch "$(cat configmap.yaml)"
```
If the ConfigMap does not exist, create it:
```
kubectl apply -f configmap.yaml
```

(Optional) Tune isolation percentages per QoS class

To adjust L3 cache and memory bandwidth per QoS class, update the ConfigMap with explicit percentages:

apiVersion: v1
kind: ConfigMap
metadata:
  name: ack-slo-config
  namespace: kube-system
data:
  resource-qos-config: |
    {
      "clusterStrategy": {
        "lsClass": {
          "resctrlQOS": {
            "enable": true,
            "catRangeEndPercent": 100,  # LS pods: 100% of L3 cache (default)
            "mbaPercent": 100           # LS pods: 100% memory bandwidth (default)
          }
        },
        "beClass": {
          "resctrlQOS": {
            "enable": true,
            "catRangeEndPercent": 30,   # BE pods: 30% of L3 cache (default)
            "mbaPercent": 100           # BE pods: 100% memory bandwidth (default)
          }
        }
      }
    }

The resctrlQOS parameters:

Parameter	Type	Valid values	Default (LS class)	Default (BE class)	Description
`enable`	Boolean	`true` / `false`	—	—	Enables or disables L3 cache and MBA isolation.
`catRangeEndPercent`	Int	0–100 (%)	100	30	Percentage of the L3 cache allocated to the QoS class.
`mbaPercent`	Int	0–100 (%), must be a multiple of 10	100	100	Percentage of memory bandwidth available to the QoS class.

Label pods with a QoS class

Apply the koordinator.sh/qosClass label to pods you want to isolate. This example creates a BE pod running a memory stress workload:

apiVersion: v1
kind: Pod
metadata:
  name: pod-demo
  labels:
    koordinator.sh/qosClass: 'BE'  # Assigns this pod to the BE QoS class.
spec:
  containers:
  - name: pod-demo
    image: polinux/stress
    resources:
      requests:
        cpu: 1
        memory: "50Mi"
      limits:
        cpu: 1
        memory: "1Gi"
    command: ["stress"]
    args: ["--vm", "1", "--vm-bytes", "256M", "-c", "2", "--vm-hang", "1"]

Note For Deployments, set the koordinator.sh/qosClass label in the template.metadata field, not at the Deployment level.

Deploy the pod:

kubectl apply -f pod-demo.yaml