Container Service for Kubernetes:Koordinator Descheduler and Kubernetes Descheduler

Before you begin

Familiarize yourself with the features, API attributes, and policy configurations (system, template, policy plugin, and evictor plugin) of the Kubernetes Descheduler before comparing it with Koordinator Descheduler.

Relationship between Koordinator Descheduler and Kubernetes Descheduler

Before version 0.25.0, the Kubernetes Descheduler community proposed the Descheduling Framework, a plug-in-based approach to manage the descheduling lifecycle, with two goals:

• Transform descheduling policies and pod eviction logic into plug-ins for greater flexibility.

• Implement a runtime that supports plug-in loading, execution, and descheduling lifecycle management.

With Kubernetes Descheduler 0.26.0, Koordinator Descheduler v1.2.0 integrated all policies and the DefaultEvictor plug-in, and added the MigrationController evictor.

Feature differences

Koordinator Descheduler is compatible with most Kubernetes Descheduler use cases but differs in configuration methods, descheduling policies, evictors, and eviction control.

1. Koordinator Descheduler began developing its Descheduling Framework before Kubernetes Descheduler and uses a different configuration API: v1alpha2/DeschedulerConfiguration.

2. Koordinator Descheduler adds the LowNodeLoad policy plug-in for hotspot-aware descheduling based on actual node utilization, providing more accurate results than the native LowNodeUtilization plug-in that uses resource allocation rates. See Use hotspot-aware descheduling.

3. Koordinator Descheduler adds the MigrationController evictor. MigrationController offers safer and more extensive eviction control. See Evictor plug-in configuration.

Configuration examples

The following examples use the RemovePodsViolatingNodeTaints policy (RemovePodsViolatingNodeTaints) to compare the configuration formats of Kubernetes Descheduler and Koordinator Descheduler. The configuration version is defined by the apiVersion and kind fields.

• Kubernetes Descheduler configuration example

  • For versions earlier than v0.27.0, the configuration version is descheduler/v1alpha1/DeschedulerPolicy.

    # API attributes.
    apiVersion: "descheduler/v1alpha1"
    kind: "DeschedulerPolicy"
    
    # System configurations.
    nodeSelector: "node=node1"
    maxNoOfPodsToEvictPerNode: 10
    maxNoOfPodsToEvictPerNamespace: 10
    
    # Eviction configurations. In the other two configuration APIs, these parameters are part of the evictor plug-in configuration.
    evictFailedBarePods: false
    evictLocalStoragePods: true
    evictSystemCriticalPods: true
    evictDaemonSetPods: false
    ignorePvcPods: false
    
    # Policy configurations. In the other two APIs, these are provided as descheduling templates and split into plug-in activation and parameter settings.
    strategies:
      "RemovePodsViolatingNodeTaints":
        enabled: true
        params:
          nodeFit: true # The nodeFit parameter must be enabled in each policy. In the other two APIs, nodeFit is part of the evictor plug-in configuration.
          excludedTaints:
            - deschedule=not-allow # Ignore nodes with the taint key="deschedule" and value="not-allow".

  • For v0.27.0 and later, the configuration version is descheduler/v1alpha2/DeschedulerPolicy.

    # API attributes.
    apiVersion: "descheduler/v1alpha2"
    kind: "DeschedulerPolicy"
    
    # System configurations.
    nodeSelector: "node=node1" 
    maxNoOfPodsToEvictPerNode: 10
    maxNoOfPodsToEvictPerNamespace: 10
    # End of system configurations.
    
    # A list of descheduling templates.
    profiles:
    - name: kubernetes-descheduler  # Specify the name of the descheduling template.
        
      # Template configurations. 
      plugins:      
        deschedule:
          enabled:
            - "RemovePodsViolatingNodeTaints" # The data type of the `enabled` field is a list of strings.
        balance:
          disabled: 
            - "*"
      # End of template configurations.         
          
      # A list of plug-in configurations.
      pluginConfig:
        
        # Configuration for the RemovePodsViolatingNodeTaints policy plug-in.    
        - name: RemovePodsViolatingNodeTaints # Node taint validation plug-in configuration.
          args:
            excludedTaints: 
            - deschedule=not-allow # Ignore nodes with the taint key="deschedule" and value="not-allow".
        
        # Configuration for the DefaultEvictor evictor plug-in.  
        - name: "DefaultEvictor"
          args:                
            evictFailedBarePods: false
            evictLocalStoragePods: true
            evictSystemCriticalPods: true
            evictDaemonSetPods: false
            ignorePvcPods: false
            nodeFit: true # The nodeFit parameter is part of the evictor plug-in configuration.

• Koordinator Descheduler configuration example

  The configuration version is descheduler/v1alpha2/DeschedulerConfiguration.

  # API attributes.
  apiVersion: descheduler/v1alpha2
  kind: DeschedulerConfiguration
  
  # System configurations.
  dryRun: false  
  deschedulingInterval: 120s 
  nodeSelector: 
    node: node1 # The data type of nodeSelector is different from the other two configuration APIs.
  maxNoOfPodsToEvictPerNode: 10 
  maxNoOfPodsToEvictPerNamespace: 10 
  # End of system configurations.
  
  # A list of descheduling templates.            
  profiles:
  - name: koord-descheduler
    
    # Template configurations.
    plugins:
      deschedule: 
        enabled: 
          - name: RemovePodsViolatingNodeTaints # The data type of the `enabled` field is a list of structs.          
      balance: 
        disabled: 
          - name: "*" 
      evict: # Use the `evict` field to specify the evictor to use.          
        enabled:
          - name: MigrationController # MigrationController is enabled by default. 
          # - name: DefaultEvictor # Optional: DefaultEvictor. 
      filter: # Use the `filter` field to specify the eviction filter to use. 
        enabled:
          - name: MigrationController # MigrationController is enabled by default.
          # - name: DefaultEvictor # Optional: DefaultEvictor.
    # End of template configurations.
            
    # A list of plug-in configurations.
    pluginConfig:
      
      # Configuration for the RemovePodsViolatingNodeTaints policy plug-in.    
      - name: RemovePodsViolatingNodeTaints # Node taint validation plug-in configuration.
        args:
          excludedTaints: 
          - deschedule=not-allow # Ignore nodes with the taint key="deschedule" and value="not-allow".
      
      # Configuration for the DefaultEvictor evictor plug-in.  
      - name: "DefaultEvictor"
        args:                
          evictFailedBarePods: false
          evictLocalStoragePods: true
          evictSystemCriticalPods: true
          evictDaemonSetPods: false
          ignorePvcPods: false
          nodeFit: true # The nodeFit parameter is part of the evictor plug-in configuration.

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsViolatingNodeAffinity"
      args:
        nodeAffinityType:
        - "preferredDuringSchedulingIgnoredDuringExecution"
    plugins:
      deschedule:
        enabled:
          - "RemovePodsViolatingNodeAffinity"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsViolatingTopologySpreadConstraint"
      args:
        constraints:
          - DoNotSchedule
        topologyBalanceNodeFit: false  
    plugins:
      balance:
        enabled:
          - "RemovePodsViolatingTopologySpreadConstraint"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemovePodsHavingTooManyRestarts"
      args:
        states:
        - "CrashLoopBackOff"
    plugins:
      deschedule:
        enabled:
          - "RemovePodsHavingTooManyRestarts"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "PodLifeTime"
      args:
        states:
        - "CrashLoopBackOff"
        - "ImagePullBackOff"
    plugins:
      deschedule:
        enabled:
          - "PodLifeTime"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "RemoveFailedPods"
      args:
        exitCodes:
        - 1
    plugins:
      deschedule:
        enabled:
          - "RemoveFailedPods"

apiVersion: "descheduler/v1alpha2"
kind: "DeschedulerPolicy"
profiles:
  - name: ProfileName
    pluginConfig:
    - name: "DefaultEvictor"
      args:
        evictDaemonSetPods: false
        minReplicas: 2
    - name: "PodLifeTime"
      args:
        maxPodLifeTimeSeconds: 86400
    plugins:
      deschedule:
        enabled:
          - "PodLifeTime"

Migrating to Koordinator Descheduler

To migrate from Kubernetes Descheduler to Koordinator Descheduler, follow these version-specific steps:

1. Uninstall your current Kubernetes Descheduler. Running multiple deschedulers in a cluster causes unpredictable race conditions.

2. Install and configure Koordinator Descheduler as described in Enable the descheduling feature.

   1. Configure system-level settings in Koordinator Descheduler based on System configuration differences.

   2. Create a descheduling template compatible with Koordinator Descheduler.

      • If you use Kubernetes Descheduler 0.26.0 or earlier, only the v1alpha1/DeschedulerPolicy API is supported. Complete the configuration based on Template configuration differences.

      • If you use Kubernetes Descheduler later than 0.26.0, confirm which configuration API you use.

        • If the API is v1alpha1/DeschedulerPolicy, create a descheduling template compatible with Koordinator Descheduler based on Template configuration differences.

        • If the API is v1alpha2/DeschedulerPolicy, modify the configuration fields in Koordinator Descheduler based on Template configuration differences.

   3. Configure plug-ins in Koordinator Descheduler based on Policy plug-in configuration differences and Evictor plug-in configuration differences.

References

• Install the ack-koordinator component and enable descheduling: Enable the descheduling feature.

  You can also configure advanced parameters for Koordinator Descheduler, descheduling templates, descheduling policy plug-ins, and evictor plug-ins in a ConfigMap. For more information, see Configure advanced parameters.

• ack-koordinator component overview and change log: ack-koordinator (FKA ack-slo-manager).