ack-slo-manager is a key component used by Container Service for Kubernetes (ACK) clusters to support service level objective (SLO)-aware workload scheduling. ack-slo-manager improves resource utilization and ensures the performance of your applications. This topic introduces ack-slo-manager and describes the usage notes and release notes for ack-slo-manager.

Prerequisites

ack-slo-manager supports only ACK clusters whose version is 1.18.0 or later.

Introduction

ack-slo-manager is a key component used by ACK clusters to support SLO-aware workload scheduling. SLO-aware workload scheduling is a method to schedule different types of workloads to the same node. The workloads run at different resource demand levels to meet SLOs. This improves resource utilization and ensures the performance of the workloads. The following table describes the scenarios in which ack-slo-manager can be used.
Scenario References
CPU scheduling
Load-aware scheduling Load-aware pod scheduling
Fine-grained scheduling

ack-slo-manager is an optimized version of resource-controller and supports all features of resource-controller. resource-controller will soon be deprecated. If you are using resource-controller, follow the steps described in the Upgrade to ack-slo-manager section to upgrade from resource-controller to ack-slo-manager.

Manage components

ack-slo-manager is available on the Add-ons page of the Container Service for Kubernetes (ACK) console. You can go to the Add-ons page to install, update, and uninstall ack-slo-manager. If the version of ack-slo-manager that you are using is lower than V0.7, ack-slo-manager is installed from the marketplace. Follow the steps described in Migrate ack-slo-manager from the marketplace to the Add-ons page to migrate ack-slo-manager from the marketplace to the Add-ons page.

Install ack-slo-manager

  1. Log on to the ACK console and click Clusters in the left-side navigation pane.
  2. On the Clusters page, click More > Manage Components in the Actions column of the cluster that you want to manage.
  3. On the Add-ons page, find and click ack-slo-manager, and then click Install on the ack-slo-manager card.
  4. In the Install ack-slo-manager dialog box, configure parameters and click OK.
  5. In the left-side navigation pane, choose Applications > Helm to view the status of ack-slo-manager.
    If the Status column of ack-slo-manager displays Deployed, ack-slo-manager is deployed.

Modify ack-slo-manager

  1. Click Configuration on the ack-slo-manager card of the Add-ons page.
  2. In the ack-slo-manager Parameters dialog box, modify the parameters and then click OK.
    ACK will re-install ack-slo-manager based on the modified configurations.

Update ack-slo-manager

  1. Click Upgrade on the ack-slo-manager card of the Add-ons page.
  2. In the Update message, confirm the information and then click OK.
    Important
    • The Upgrade button is displayed only if the component is not updated to the latest version.
    • The update overwrites all changes that are made to the Deployment and DaemonSet of ack-slo-manager by using other methods.

Uninstall ack-slo-manager

The topology-aware CPU scheduling feature creates a topology ConfigMap in the kube-system namespace for each node in an ACK cluster. ack-slo-manager 0.5.1 and later can automatically delete the topology ConfigMaps of the nodes that are removed from a cluster. If you uninstall ack-slo-manager, the topology ConfigMaps of the existing nodes in the cluster are still retained. The retained topology ConfigMaps do not impact other features but occupy storage space. We recommend that you delete these ConfigMaps at your earliest convenience.

  1. On the ack-slo-manager card of the Add-ons page, click Uninstall. In the Uninstall message, click OK.
  2. Delete topology ConfigMaps.
    1. In the left-side navigation pane, choose Configurations > ConfigMaps. In the upper part of the page, select the kube-system namespace.
    2. Enter -numa-info into the Name search box and select a ConfigMap that matches the ${NODENAME}-numa-info naming rule from the list. Then, click Delete in the Actions column to delete the ConfigMap.
    3. In the Confirm message, click OK.

Migrate components

Migrate ack-slo-manager from the marketplace to the Add-ons page

If you have modified the ConfigMap of ack-slo-manager in the marketplace, back up the ConfigMap before you update ack-slo-manager.

  1. Optional:You can use one of the following methods to back up the ConfigMap of ack-slo-manager:
    • Method 1: Use kubectl to back up the ConfigMap
      1. Run the kubectl get cm -n kube-system ack-slo-manager-config -o yaml > slo-config.yaml command to save the ConfigMap to a file named slo-config.yaml. Replace kube-system with the actual namespace of the ConfigMap and ack-slo-manager-config with the actual name of the ConfigMap.
      2. Run the vim slo-config.yaml command to modify the preceding file. Set namespace in the ConfigMap to kube-system, set name to ack-slo-config, and delete all annotations and labels in the ConfigMap in case the current configurations are overwritten during the update.
      3. Run the kubectl apply -f slo-config.yaml command to apply the modified ConfigMap.
    • Method 2: Use the ACK console to back up the ConfigMap
      1. Record the key-value pair in the ConfigMap.
        1. In the left-side navigation pane, choose Configurations > ConfigMaps. In the top navigation bar of the page, select the namespace that is specified when you install ack-slo-manager from the marketplace. The default namespace is kube-system.
        2. Enter the ConfigMap name ack-slo-manager-config into the search box, click the name of the ConfigMap that is displayed, and record the key-value pair.
      2. Use the recorded key-value pair to create another ConfigMap.
        1. In the left-side navigation pane, choose Configurations > ConfigMaps. In the top navigation bar, select All Namespaces.
        2. In the upper-right corner of the Configurations page, click Create. In the panel that appears, enter ack-slo-config into the ConfigMap Name field, select the kube-system namespace, click + Add, enter the recorded key-value pair, and then click OK.
  2. Update ack-slo-manager to the latest version. For more information, see Update ack-slo-manager.
    Important If you have modified the ConfigMap of ack-slo-manager in the marketplace, you need to specify the name of the backup ConfigMap that is created in Step 1 in the ack-slo-manager Parameters dialog box.

Upgrade from resource-controller to ack-slo-manager

ack-slo-manager supports all features of resource-controller. If you have installed resource-controller in your cluster, perform the following steps to upgrade from resource-controller to ack-slo-manager.

  1. Update resource-controller to the latest version.
    1. Log on to the ACK console and click Clusters in the left-side navigation pane.
    2. On the Clusters page, find the cluster that you want to manage and click More > Manage Components in the Actions column of the cluster.
    3. On the Add-ons page, find and click resource-controller and then click Upgrade on the resource-controller card.
    4. In the message that appears, click OK.
  2. Install and configure ack-slo-manager.
    1. On the Add-ons page, find and click ack-slo-manager and then click Install on the ack-slo-manager card.
    2. In the Install ack-slo-manager dialog box, modify the agentFeatureGates parameter to enable or disable the feature gates of ack-slo-agent.
      1. Check whether the cluster has the CPU Burst feature described in Dynamically modify the resource parameters of a pod enabled. This feature can create a CustomResourceDefinition (CRD) or pod annotation to modify the cgroup configuration file named cpu.cfs_quota_us. If this feature is enabled, perform Step ii. If not, perform Step c.
      2. Run the following command to query the feature-gate configurations from the YAML file of the ack-slo-agent DaemonSet:
        kubectl get daemonset -n kube-system ack-slo-agent -o yaml |grep feature-gates
        - --feature-gates=AllAlpha=false,AllBeta=false,...,CPUBurst=true,....
      3. Modify the feature-gate configurations based on the following content. The new configurations disable CPU Burst (CPUBurst=false). Separate multiple parameters with commas (,).

        After CPU Burst is disabled, CPU Burst is unavailable for all containers in the cluster. This prevents the Cgroup configuration file cpu.cfs_quota_us from being modified by two modules at the same time.

        AllAlpha=false,AllBeta=false,...,CPUBurst=false,....
      4. If you want to dynamically scale the CPU resources of containers, we recommend that you enable the CPU Burst feature. This feature can automatically modify the CPU limit of pods. For more information, see CPU Burst.
    3. Configure other parameters based on your business requirements and then click OK.
    4. In the left-side navigation pane of the Clusters page, choose Applications > Helm to check the status of ack-slo-manager.
      If the Status column of ack-slo-manager displays Deployed, ack-slo-manager is deployed.
  3. Uninstall resource-controller.
    1. On the Add-ons page, find and click resource-controller and then click Uninstall on the resource-controller card.
    2. In the Uninstall message, click OK.

FAQ

Component installation error: no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1" ensure CRDs are installed first

Prometheus is not installed in the cluster. Refer to Enable ARMS Prometheus to install the Prometheus component or clear Enable Prometheus Metrics for ack-slo-manager when you install ack-slo-manager.

Release notes

November 2022

Version Image address Release date Description Impact
v0.8.0
  • registry.cn-hangzhou.aliyuncs.com/acs/koord-manager:v0.8.0
  • registry.cn-hangzhou.aliyuncs.com/acs/koordlet:v0.8.0
2022-11-17
  • Load-aware pod scheduling is updated to support Kubernetes 1.22.
  • Internal API operations are optimized.
If you want to use load-aware pod scheduling after ack-slo-manager is updated, you must update the Kubernetes version of your cluster to 1.22.15-ack-2.0. Other features are not affected.

September 2022

Version Image address Release date Description Impact
v0.7.2 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.7.2 2022-09-16 The following issue induced by V0.7.1 is fixed: topology-aware scheduling does not take effect on pods. No impact on workloads
v0.7.1 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.7.1 2022-09-02
  • The CPU throttling issue that occurs when topology-aware scheduling is used in a CentOS 3 kernel environment is fixed.
  • Prometheus Monitoring can be enabled and disabled.
  • Resource profiling features are supported.
  • ack-slo-manager can no longer be installed from the marketplace in the ACK console.
No impact on workloads

August 2022

Version Image address Release date Description Impact
v0.7.0 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.7.0 2022-08-08 ack-slo-manager is migrated from the marketplace to the Add-ons page in the ACK console. If you want to install ack-slo-manager, go to the Add-ons page. No impact on workloads

July 2022

Version Image address Release date Description Impact
v0.6.0 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.6.0 2022-07-26 Internal API operations are optimized and component configurations are simplified. No impact on workloads

June 2022

Version Image address Release date Description Impact
v0.5.2 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.5.2 2022-06-14
  • Internal API operations are optimized.
  • The CPU Quality of Service (QoS) feature is optimized. For more information, see CPU QoS.
  • Automatic creation of the ack-slo-manager namespace can be disabled.
No impact on workloads
v0.5.1 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.5.1 2022-06-02
  • The CPU QoS feature is supported. For more information, see CPU QoS.
  • Internal API operations are optimized. The topology ConfigMaps of the nodes that are removed from a cluster can be automatically deleted.
No impact on workloads

April 2022

Version Image address Release date Description Impact
v0.5.0 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.5.0 2022-04-29 No impact on workloads
v0.4.1 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.4.1 2022-04-14 No impact on workloads
v0.4.0 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.4.0 2022-04-11 The memory consumption of slo-agent is reduced. No impact on workloads

February 2022

Version Image address Release date Description Impact
v0.3.0 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.3.0 2022-02-25 No impact on workloads

December 2021

Version Image address Release date Description Impact
v0.2.0 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.2.0 2021-12-10 No impact on workloads

September 2021

Version Image address Release date Description Impact
v0.1.1 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.1.1-c2ccefa 2021-09-02 Internal API operations are optimized. No impact on workloads

July 2021

Version Image address Release date Description Impact
v0.1.0 registry.cn-hangzhou.aliyuncs.com/acs/ack-slo-manager:v0.1.0-09766de 2021-07-08 Workload-aware scheduling is supported. For more information, see Load-aware pod scheduling. No impact on workloads