All Products
Search

This Product

    Container Service for Kubernetes:Implement elastic scaling for ECI nodes with the Ray autoscaler

    Document Center

    Container Service for Kubernetes:Implement elastic scaling for ECI nodes with the Ray autoscaler

    Last Updated:Jun 17, 2026

    Scale Ray workers to serverless ECI pods on demand — no node provisioning, pay-per-use.

    How it works

    Autoscaling in a KubeRay cluster operates at three levels:

    • Ray actor/task: Some Ray libraries (such as Ray Serve) automatically adjust actor count based on request volume.

    • Ray node: The Ray autoscaler adjusts Ray pod count based on *logical* resource demand in @ray.remote annotations — not physical CPU or memory utilization. When a task or actor requests more resources than available, the autoscaler queues the request and provisions new worker pods. Idle nodes are removed over time.

    • Kubernetes node: When the Kubernetes cluster lacks capacity for new Ray pods, the Kubernetes autoscaler provisions additional nodes. This document covers Ray-level autoscaling to ECI virtual nodes.

    With ECI, each new Ray worker pod maps to an ECI instance rather than an Elastic Compute Service (ECS) node — fast, isolated startup without managing infrastructure.

    Cluster architecture

    image.png

    Prerequisites

    Ensure that you have:

    Configure and verify ECI elastic scaling

    Verify the virtual-kubelet node

    Confirm that the virtual-kubelet virtual node is present in your cluster:

    kubectl get node

    Expected output:

    NAME                            STATUS   ROLES    AGE   VERSION
cn-hangzhou.172.XX.XX.20         Ready    <none>   19h   v1.26.3-aliyun.1
cn-hangzhou.172.XX.XX.236        Ready    <none>   82m   v1.26.3-aliyun.1
cn-hangzhou.172.XX.XX.41         Ready    <none>   19h   v1.26.3-aliyun.1
virtual-kubelet-cn-hangzhou-k   Ready    agent    16m   v1.26.3-aliyun.1

    The virtual-kubelet-cn-hangzhou-k node is the ECI virtual node. Worker pods on this node run as ECI instances.

    Create the values.yaml configuration file

    Create a values.yaml file that labels the worker group for ECI scheduling:

    cat > values.yaml <<EOF
worker:
  groupName: workergroup
  labels:
    alibabacloud.com/eci: "true"
EOF

    The alibabacloud.com/eci: "true" label instructs the scheduler to place worker pods on the ECI virtual node.

    Deploy the Ray cluster with ECI support

    Redeploy your Ray cluster with the updated configuration:

    helm uninstall ${RAY_CLUSTER_NAME} -n ${RAY_CLUSTER_NS}
helm install ${RAY_CLUSTER_NAME} aliyunhub/ack-ray-cluster -n ${RAY_CLUSTER_NS} -f values.yaml

    Verify the head pod is running

    Confirm the head pod is ready:

    kubectl get pod

    Expected output:

    NAME                                           READY   STATUS    RESTARTS   AGE     IP             NODE                            NOMINATED NODE   READINESS GATES
myfirst-ray-cluster-head-7fgp4                 2/2     Running   0          7m2s    172.16.0.241   cn-hangzhou.172.16.0.240        <none>           <none>

    The head pod runs on a regular ECS node. Its --num-cpus defaults to 0, so the Ray autoscaler routes all compute tasks to worker pods.

    Log in to the head node

    Replace the pod name with your actual head pod name:

    kubectl -n ${RAY_CLUSTER_NS} exec -it myfirst-ray-cluster-head-7fgp4 -- bash

    Submit a Python job to trigger scale-out

    This script submits two tasks, each requesting 1 vCPU. Since the head pod has --num-cpus=0 and each worker pod defaults to 1 vCPU and 1 GB memory, the Ray autoscaler provisions two new ECI worker pods.

    Note

    The Ray autoscaler scales on *logical* resource requests in @ray.remote, not physical CPU or memory utilization. The num_cpus=1 declaration triggers scale-out — not actual CPU usage.

    import time
import ray
import socket

ray.init()

@ray.remote(num_cpus=1)
def get_task_hostname():
    time.sleep(120)
    host = socket.gethostbyname(socket.gethostname())
    return host

object_refs = []
for _ in range(2):
    object_refs.append(get_task_hostname.remote())

ray.wait(object_refs)

for t in object_refs:
    print(ray.get(t))

    Verify scale-out

    Confirm that two ECI worker pods are provisioned on the virtual node:

    kubectl get pod -o wide

    Expected output:

    NAME                                           READY   STATUS     RESTARTS   AGE     IP             NODE                            NOMINATED NODE   READINESS GATES
myfirst-ray-cluster-head-7fgp4                 2/2     Running    0          4m56s   172.16.0.241   cn-hangzhou.172.16.0.240        <none>           <none>
myfirst-ray-cluster-worker-workergroup-6s2cl   0/1     Init:0/1   0          4m5s    172.16.0.17    virtual-kubelet-cn-hangzhou-k   <none>           <none>
myfirst-ray-cluster-worker-workergroup-l9qgb   1/1     Running    0          4m5s    172.16.0.16    virtual-kubelet-cn-hangzhou-k   <none>

    Both worker pods run on virtual-kubelet-cn-hangzhou-k, confirming they are ECI instances. Init:0/1 means the pod is initializing — it transitions to Running when the ECI instance is ready.

    After tasks complete and pods go idle, the Ray autoscaler terminates the worker pods.

    Next steps