Automate Pod Scheduling to ECI with eci-profile - ACK

Managing pod scheduling in a mixed cluster—where real nodes and virtual nodes coexist—normally requires editing the YAML for every workload you want to run on Elastic Container Instance (ECI). An ECI profile eliminates this overhead by centralizing scheduling rules, annotation injection, and cluster-level network settings in a single ConfigMap, letting you control ECI pod behavior without touching individual workload definitions.

When a pod is created, ack-virtual-node reads the ConfigMap named eci-profile in the kube-system namespace and applies its settings automatically. Two mechanisms handle pod-level behavior:

ECI Scheduler — routes pods to ECI based on pod or namespace labels, using a mutating webhook instead of explicit node selectors in workload YAML files.
ECI Effect — injects annotations and labels into matched pods at creation time, enabling ECI features such as GPU instance types, image caches, and NTP configuration.

Cluster-level parameters in the profile (cluster IP address, hybrid cloud, log collection, and vSwitches) support hot updates: changes take effect immediately for new ECI pods without restarting ack-virtual-node. Existing pods pick up changes only after a rolling update.

Prerequisites

Before you begin, make sure that:

The ack-virtual-node component in the cluster is up to date. For upgrade instructions, see Manage components.
Mutating webhooks are enabled in the cluster (required for ECI Scheduler). Pods in ACK Serverless clusters are automatically scheduled to ECI and do not need ECI Scheduler.

View the current ECI profile

Run the following command to inspect the current eci-profile ConfigMap:

kubectl get cm -n kube-system eci-profile -o yaml

The output looks similar to:

apiVersion: v1
data:
  enableClusterIp: "true"
  enableHybridMode: "false"
  enableLinuxArm64Node: "false"
  enableLogController: "false"
  enablePVCController: "false"
  enablePrivateZone: "false"
  enableReuseSSLKey: "false"
  featureGates: "WaitForFirstConsumer=false"
  securityGroupId: sg-2zeeyaaxlkq9sppl****
  selectors: ""
  slsMachineGroup: ""
  vSwitchIds: vsw-2ze23nqzig8inprou****,vsw-2ze94pjtfuj9vaymf****
  vpcId: vpc-2zeghwzptn5zii0w7****
kind: ConfigMap
metadata:
  creationTimestamp: "2023-01-11T08:28:14Z"
  name: eci-profile
  namespace: kube-system
  resourceVersion: "356"
  uid: b345fa8c-919e-41fc-a981-57864b1a****

The data section contains two types of parameters:

Parameter	Purpose	Details
`selectors`	Configure ECI Scheduler and ECI Effect	Configure selectors
All other parameters	Cluster-level settings (VPC, vSwitches, log collection, etc.)	Update cluster-level parameters

Edit the ECI profile

Use either of the following methods to modify the eci-profile ConfigMap.

kubectl

kubectl edit configmap eci-profile -n kube-system

ACK console

Log on to the Container Service ACK console.
On the Clusters page, click the name of the target cluster.
In the left-side navigation pane, choose Configurations > ConfigMaps.
Select kube-system from the Namespace drop-down list.
Find eci-profile and click Edit YAML in the Actions column.

Configure selectors

Selectors define which pods ECI Scheduler routes to ECI and which annotations or labels ECI Effect injects into those pods.

Selector fields

Each selector entry supports the following fields:

Field	Required	Type	Description
`name`	Yes	String	A unique identifier for this selector
`namespaceSelector.matchLabels`	No	Map	Namespace labels to match; multiple labels use AND logic
`objectSelector.matchLabels`	No	Map	Pod labels to match; multiple labels use AND logic
`effect.annotations`	No	Map	Annotations to inject into matched pods
`effect.labels`	No	Map	Labels to inject into matched pods

Selector matching rules

Include at least one of namespaceSelector or objectSelector in each selector. If both are configured, a pod must satisfy both filters to match.
If neither namespaceSelector nor objectSelector is set but effect is configured, the effect applies to all pods scheduled to ECI.
When multiple selectors match a pod, they are evaluated in order. Injected annotations and labels do not overwrite existing ones on the pod. Priority (descending): existing pod annotations/labels > effect of the first matching selector > effect of the second matching selector > and so on.

Configuration template

Remove all comments before applying the configuration.

data:
  selectors: |
    [
      {
        "name": "selector-demo1",        # Required. Unique selector name.
        "namespaceSelector": {            # Optional. Matches by namespace labels.
          "matchLabels": {               # AND logic across multiple labels.
            "eci": "true"
          }
        },
        "objectSelector": {              # Optional. Matches by pod labels.
          "matchLabels": {               # AND logic across multiple labels.
            "eci": "true"
          }
        },
        "effect": {                      # Optional. Annotations and labels to inject.
          "annotations": {
            "k8s.aliyun.com/eci-use-specs": "ecs.c6.xlarge"
          },
          "labels": {
            "created-by-eci": "true"
          }
        }
      },
      {
        "name": "selector-demo2",
        "objectSelector": {
          "matchLabels": {
            "eci": "test"
          }
        }
      }
    ]

Verify selectors

After saving the profile, confirm the selectors are active:

kubectl get mutatingwebhookconfigurations -o yaml vk-webhook

If the output contains your configured selectors, they are active. If not, check the selector JSON for formatting errors.

Configuration examples

Example 1: Route specific pods to ECI

The following selector routes a pod to ECI when the pod has the created-by-eci: true label and its namespace has the type: eci label.

data:
  selectors: |
    [
      {
        "name":"eci-selector",
        "namespaceSelector":{
          "matchLabels":{
            "type":"eci"
          }
        },
        "objectSelector":{
          "matchLabels":{
            "created-by-eci":"true"
          }
        }
      }
    ]

Example 2: Route pods to ECI with a GPU instance type

The following selector routes pods in GPU-labeled namespaces to ECI using the ecs.gn6v-c8g1.2xlarge GPU-accelerated ECS instance type, and appends the gpu: test label.

data:
  selectors: |
    [
      {
        "name":"gpu-namespace-selector",
        "namespaceSelector":{
          "matchLabels":{
            "gpu":"true"
          }
        },
        "effect": {
          "annotations": {
            "k8s.aliyun.com/eci-use-specs":"ecs.gn6v-c8g1.2xlarge"
          },
          "labels":{
            "gpu":"test"
          }
        }
      }
    ]

Example 3: Route pods to ECI with automatic image cache matching

The following selector routes pods with the imc: auto label to ECI and enables automatic image cache matching.

data:
  selectors: |
    [
      {
        "name":"autoimc-object-selector",
        "objectSelector":{
          "matchLabels":{
            "imc":"auto"
          }
        },
        "effect": {
          "annotations": {
            "k8s.aliyun.com/eci-auto-imc": "true"
          }
        }
      }
    ]

Update cluster-level parameters

The following parameters in the data section control cluster-wide behavior. Changes take effect immediately for new ECI pods without restarting ack-virtual-node. Existing pods pick up changes only after a rolling update.

If you do not set these parameters when creating an ECI pod, the eci-profile values are used as defaults.

Parameter	Default	Description
`enableClusterIp`	`"true"`	Enables cluster IP address support
`enableHybridMode`	`"false"`	Enables hybrid cloud mode
`enableLinuxArm64Node`	`"false"`	Enables ARM-based virtual nodes. See Schedule pods to an ARM-based virtual node
`enableLogController`	`"false"`	Enables log collection using a Custom Resource Definition (CRD) for Simple Log Service (SLS). When set to `"true"`, also set `slsMachineGroup`
`enablePVCController`	`"false"`	Enables online disk extension, allowing hot extension on PersistentVolumeClaims (PVCs) bound to disks
`enablePrivateZone`	`"false"`	Enables PrivateZone for domain name resolution
`enableReuseSSLKey`	`"false"`	Reuses the same SSL certificate across all pods instead of issuing one per pod. Improves pod creation speed at the cost of per-pod certificate isolation. See SSL key reuse
`featureGates`	`"WaitForFirstConsumer=false"`	Controls canary features. Only `WaitForFirstConsumer` is supported. See WaitForFirstConsumer mode
`securityGroupId`	—	Security group for ECI pods
`slsMachineGroup`	`""`	SLS machine group for log collection. Required when `enableLogController` is `"true"`
`vSwitchIds`	—	Comma-separated vSwitch IDs for ECI pods
`vpcId`	—	VPC for ECI pods

SSL key reuse

By default, ack-virtual-node issues a unique SSL certificate to each pod. When creating a large number of pods, this can slow pod creation. Setting enableReuseSSLKey to "true" causes ack-virtual-node to issue the same SSL certificate to all pods, improving creation speed.

Important

Enabling SSL key reuse reduces per-pod certificate isolation. Evaluate this trade-off before enabling in production.

WaitForFirstConsumer mode

When featureGates is set to "WaitForFirstConsumer=true", the StorageClass uses WaitForFirstConsumer volume binding mode.

In this mode, PersistentVolume (PV) and backend storage creation is deferred until a pod using the PersistentVolumeClaim (PVC) is scheduled. The zone and region of the scheduled node—not those specified in the StorageClass—are used to create storage resources, ensuring computing and storage resources land in the same zone.

Before enabling:

Update the csi-provisioner component to the latest version.

For more information about volume binding modes, see Volume binding mode.

What's next

ECI Pod Annotation — full list of annotations for configuring ECI pods
Manage components — keep ack-virtual-node up to date