Container Service for Kubernetes:Colocate an online service with a video transcoding application

Mode 1: Exclusive deployment (baseline)

Only the NGINX service runs on the tested machine.

1. Deploy NGINX as described in Deploy the NGINX service and wrk.

2. Send load from the stress test machine:

   # Replace node_ip with the IP address of the tested machine.
   ./wrk -t6 -c54 -d60s --latency http://${node_ip}:8000/

3. Check CPU utilization:

   kubectl top node

   Expected output:

   NAME                      CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
   cn-beijing.192.168.2.93   29593m       29%    xxxx            xxxx
   cn-beijing.192.168.2.94   6874m        7%     xxxx            xxxx

   CPU utilization on the tested machine is approximately 29%.

4. After the test completes, review the wrk output. For accurate results, run multiple tests. Expected output:

   Running 1m test @ http://192.168.2.94:8000/
     6 threads and 54 connections
     Thread Stats   Avg      Stdev     Max   +/- Stdev
       Latency   402.18us    1.07ms  59.56ms   99.83%
       Req/Sec    24.22k     1.12k   30.58k    74.15%
     Latency Distribution
        50%  343.00us
        75%  402.00us
        90%  523.00us
        99%  786.00us
     8686569 requests in 1.00m, 6.88GB read
   Requests/sec: 144537.08
   Transfer/sec:    117.16MB

   The Latency Distribution section shows RT percentile values. In exclusive mode: RT-P50 is 343 microseconds, RT-P90 is 523 microseconds, and RT-P99 is 786 microseconds.

Mode 2: Default Kubernetes colocation (control)

Both NGINX and FFmpeg run on the tested machine without ack-koordinator isolation features.

Deploy NGINX as described in Deploy the NGINX service and wrk, then deploy the FFmpeg application using be-ffmpeg.yaml with the following modifications:

• Remove the koordinator.sh/qosClass: BE label.

• Remove the kubernetes.io/batch-cpu and kubernetes.io/batch-memory extended resources.

Run the wrk load test and collect CPU utilization as in Mode 1. In this control configuration, the node CPU utilization reaches approximately 65%.

Mode 3: SLO-aware colocation (experimental)

Both NGINX and FFmpeg run on the tested machine with ack-koordinator isolation features enabled.

1. Follow the Getting started guide to enable SLO-aware colocation, then configure each feature:

   • [Dynamic resource overcommitment](https://www.alibabacloud.com/help/en/document_detail/412172.html#task-2190961): Use the default configuration. This allows the system to allocate idle LS pod resources to BE pods as overcommitted batch resources (kubernetes.io/batch-cpu and kubernetes.io/batch-memory).

   • [CPU Suppress](https://www.alibabacloud.com/help/en/document_detail/268626.html#task-2088911): Set cpuSuppressThresholdPercent to 65. Use defaults for other settings. When node CPU utilization exceeds 65%, this feature throttles BE pod CPU usage to protect LS pod performance.

   • [CPU QoS](https://www.alibabacloud.com/help/en/document_detail/433810.html#task-2223861): Use the default configuration. This enables the CPU Identity capability on Alibaba Cloud Linux, giving LS pods scheduling priority over BE pods — including when simultaneous multithreading (SMT) runs threads from both pods on the same physical core.

   • [Resource isolation based on the L3 cache and MBA](https://www.alibabacloud.com/help/en/document_detail/273042.html#task-2093499): Use the default configuration. On ECS Bare Metal instances, this isolates L3 cache (last level cache) and memory bandwidth allocation (MBA) so that LS pods get priority access.

   Important

   CPU QoS requires Alibaba Cloud Linux as the node OS. L3 cache and MBA isolation requires an ECS Bare Metal instance.

2. Deploy NGINX as described in Deploy the NGINX service and wrk.

3. Create a file named besteffort-ffmpeg.yaml with the following content: Show YAML file content

   # Pod for the offline FFmpeg video transcoding application (BE QoS class, SLO-aware mode)
   apiVersion: v1
   kind: Pod
   metadata:
     name: besteffort-ffmpeg
     labels:
       app: ffmpeg
       # Set the QoS class to BE for SLO-aware scheduling.
       koordinator.sh/qosClass: BE
   spec:
     containers:
       - command:
           - start-ffmpeg.sh
           - '30'
           - '2'
           - /apps/ffmpeg/input/HD2-h264.ts
           - /apps/ffmpeg/
         image: 'registry.cn-zhangjiakou.aliyuncs.com/acs/ffmpeg-4-4-1-for-slo-test:v0.1'
         imagePullPolicy: Always
         name: ffmpeg
         resources:
           # Request dynamically overcommitted resources.
           limits:
             kubernetes.io/batch-cpu: 70k
             kubernetes.io/batch-memory: 22Gi
           requests:
             kubernetes.io/batch-cpu: 70k
             kubernetes.io/batch-memory: 22Gi
     hostNetwork: true
     restartPolicy: Never
     nodeName: cn-beijing.192.168.2.93  # Replace with the node name of your tested machine.

4. Deploy the FFmpeg application:

   kubectl apply -f besteffort-ffmpeg.yaml

5. Verify the FFmpeg pod is running:

   kubectl get pod -l app=ffmpeg -o wide

   Expected output:

   NAME                READY   STATUS    RESTARTS   AGE    IP               NODE                      NOMINATED NODE   READINESS GATES
   besteffort-ffmpeg   1/1     Running   0          15s    11.162.XXX.XXX   cn-beijing.192.168.2.93   <none>           <none>

6. Send load from the stress test machine:

   # Replace node_ip with the IP address of the tested machine.
   ./wrk -t6 -c54 -d60s --latency http://${node_ip}:8000/

7. Check CPU utilization:

   kubectl top node

   Expected output:

   NAME                      CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
   cn-beijing.192.168.2.93   65424m       63%    xxxx            xxxx
   cn-beijing.192.168.2.94   7040m        7%     xxxx            xxxx

   CPU utilization on the tested machine is approximately 63%.

8. After the test completes, review the wrk output and compare with the results from the other modes.