Use ECS instances to meet steady-state demand and use Elastic Container Instances to handle traffic spikes in Knative - Container Service for Kubernetes

Run baseline Knative workloads on existing Elastic Compute Service (ECS) instances and automatically overflow traffic spikes to Elastic Container Instances (ECI). A ResourcePolicy sets the priority: ECS first at steady state, ECI for the overflow, and ECI released first during scale-in on a best-effort basis.

Prerequisites

Before you begin, ensure that you have:

Knative deployed in your cluster. See Deploy Knative in an ACK cluster or Deploy Knative in an ACK Serverless cluster.

A kube-scheduler version that matches the Kubernetes version of your cluster:

Kubernetes version	kube-scheduler version
1.20	1.20.4-ack-7.0 or later
1.22	1.22.15-ack-2.0 or later
1.24	1.24.3-ack-2.0 or later
1.26	1.26.3-ack-4.0 or later
Later than 1.26	All versions

The ack-virtual-node add-on deployed. This add-on is required for ECI resources. See Deploy ack-virtual-node in the cluster.
A kubeconfig file and kubectl access to the cluster. See Connect to your cluster using kubectl.

Limitations

Constraint	Details
Mutually exclusive with pod deletion cost	Cannot use priority-based resource scheduling together with the pod deletion cost feature.
Mutually exclusive with ECI-based scheduling	Cannot use priority-based resource scheduling together with Elastic Container Instance-based scheduling.
Scale-in is best-effort	The ResourcePolicy uses the `prefer` strategy (BestEffort). Scale-in does not guarantee that pods are removed in strict priority order.
`max` parameter requires Kubernetes 1.22+ and scheduler 5.0+	The `max` parameter is only available when your cluster runs Kubernetes 1.22 or later and your scheduler version is 5.0 or later.
Elastic node pools must be in units without `max`	When using elastic node pools, include them in units but do not specify the `max` parameter for those units.
Pre-existing pods (scheduler < 5.0 or Kubernetes 1.20 or earlier)	Pods that existed before the ResourcePolicy was created are prioritized during scale-in.
ResourcePolicy modification (scheduler < 6.1 or Kubernetes 1.20 or earlier)	Do not modify the ResourcePolicy until all pods selected by it are deleted.

Create a ResourcePolicy for hybrid ECS and ECI scheduling

A ResourcePolicy targets a Knative Service by label and defines an ordered list of units. Each unit maps to a resource type (ECS or ECI), can optionally cap the number of pods with max, and can narrow the unit to specific nodes with nodeSelector. The scheduler works through the units in order: When an ECS unit is full or no ECS capacity is available, pods overflow to the next unit.

Create a Knative Service.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: helloworld-go
spec:
  template:
    spec:
      containers:
      - image: registry-vpc.cn-beijing.aliyuncs.com/knative-sample/helloworld-go:73fbdd56
        env:
        - name: TARGET
          value: "Knative"

Create a ResourcePolicy that targets the helloworld-go Knative Service. ECS instances have the highest priority. ECI is the fallback—pods are scheduled to ECI only when ECS capacity is exhausted or when the number of pods on an ECS unit reaches the max limit.

apiVersion: scheduling.alibabacloud.com/v1alpha1
kind: ResourcePolicy
metadata:
  name: xxx
  namespace: xxx
spec:
  selector:
    serving.knative.dev/service: helloworld-go  # Target Knative Service name
  strategy: prefer
  units:
  - resource: ecs
    max: 10           # Cap at 10 pods on this ECS node group
    nodeSelector:
      key2: value2
  - resource: ecs     # Secondary ECS node group (no cap)
    nodeSelector:
      key3: value3
  - resource: eci     # ECI absorbs overflow; released first during scale-in on a best-effort basis

Scale-in follows the reverse unit order on a best-effort basis—ECI instances are preferably released first, but strict ordering is not guaranteed.

Next steps

For more details on priority-based resource scheduling parameters, see Configure priority-based resource scheduling.