Container Service for Kubernetes:Inventory-aware cross-region multi-cluster elastic scheduling

1. Download the model

git lfs install
GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/Qwen/Qwen3-8B
cd Qwen3-8B
git lfs pull

2. Upload the model to OSS

ossutil mkdir oss://<your-bucket-name>/models/Qwen3-8B
ossutil cp -r ./Qwen3-8B oss://<your-bucket-name>/models/Qwen3-8B

3. Create a PersistentVolume and PersistentVolumeClaim in each child cluster

For detailed steps, see Use ossfs 1.0 static persistent volume.

apiVersion: v1
kind: Secret
metadata:
  name: oss-secret
stringData:
  akId: <your-oss-ak>      # AccessKey ID for accessing OSS
  akSecret: <your-oss-sk>  # AccessKey Secret for accessing OSS
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: qwen3-8b
  namespace: default
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 20Gi
  selector:
    matchLabels:
      alicloud-pvname: qwen3-8b
  storageClassName: oss
  volumeMode: Filesystem
  volumeName: qwen3-8b
---
apiVersion: v1
kind: PersistentVolume
metadata:
  labels:
    alicloud-pvname: qwen3-8b
  name: qwen3-8b
spec:
  accessModes:
    - ReadWriteMany
  capacity:
    storage: 20Gi
  csi:
    driver: ossplugin.csi.alibabacloud.com
    nodePublishSecretRef:
      name: oss-secret
      namespace: default
    volumeAttributes:
      bucket: <your-bucket-name>       # Bucket name
      otherOpts: '-o allow_other -o umask=000'
      path: <your-model-path>          # Example: /models/Qwen3-8B/
      url: <your-bucket-endpoint>      # Example: oss-cn-hangzhou-internal.aliyuncs.com
    volumeHandle: qwen3-8b
  persistentVolumeReclaimPolicy: Retain
  storageClassName: oss
  volumeMode: Filesystem

1. Create deploy.yaml

The Deployment runs 4 replicas of the Qwen3-8B model using vLLM, with each pod requesting 1 GPU.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: qwen3-8b
  name: qwen3-8b
  namespace: default
spec:
  replicas: 4
  selector:
    matchLabels:
      app: qwen3-8b
  template:
    metadata:
      labels:
        app: qwen3-8b
    spec:
      volumes:
        - name: qwen3-8b
          persistentVolumeClaim:
            claimName: qwen3-8b
        - name: dshm
          emptyDir:
            medium: Memory
            sizeLimit: 20Gi
      containers:
      - command:
        - sh
        - -c
        - vllm serve /models/qwen3-8b --port 8000 --trust-remote-code --served-model-name qwen3-8b --tensor-parallel=1 --max-model-len 8192 --gpu-memory-utilization 0.95 --enforce-eager
        image: kube-ai-registry.cn-shanghai.cr.aliyuncs.com/kube-ai/vllm-openai:v0.9.1
        name: vllm
        ports:
        - containerPort: 8000
        readinessProbe:
          tcpSocket:
            port: 8000
          initialDelaySeconds: 30
          periodSeconds: 30
        resources:
          limits:
            nvidia.com/gpu: "1"
        volumeMounts:
          - mountPath: /models/qwen3-8b
            name: qwen3-8b
          - mountPath: /dev/shm
            name: dshm

2. Create PropagationPolicy.yaml

The PropagationPolicy distributes the Deployment across two child clusters.

Key fields:

apiVersion: policy.one.alibabacloud.com/v1alpha1
kind: PropagationPolicy
metadata:
  name: demo-policy
spec:
  # Enables inventory-aware elastic scheduling
  autoScaling:
    ecsProvision: true
  preserveResourcesOnDeletion: false
  conflictResolution: Overwrite
  resourceSelectors:
  - apiVersion: apps/v1
    kind: Deployment
    name: qwen3-8b
    namespace: default
  placement:
    replicaScheduling:
      replicaSchedulingType: Divided
      weightPreference:
        dynamicWeight: AvailableReplicas
    clusterAffinity:
      clusterNames:
      - ${cluster1-id}  # Replace with your actual child cluster ID
      - ${cluster2-id}  # Replace with your actual child cluster ID

3. Apply the manifests

kubectl apply -f deploy.yaml
kubectl apply -f PropagationPolicy.yaml

After a short time, GPU node pools in both child clusters begin scaling out automatically.

Check workload scheduling status

kubectl get resourcebinding

Expected output:

NAME                  SCHEDULED   FULLYAPPLIED   OVERRIDDEN   ALLAVAILABLE   AGE
qwen3-8b-deployment   True        True           True         False          7m47s

Check pod distribution across clusters

After pods reach the Running state, run:

kubectl amc get deploy qwen3-8b -M

Expected output:

NAME       CLUSTER           READY   UP-TO-DATE   AVAILABLE   AGE     ADOPTION
qwen3-8b   cxxxxxxxxxxxxxx   2/2     2            2           3m22s   Y
qwen3-8b   cxxxxxxxxxxxxxx   2/2     2            2           3m22s   Y

All 4 replicas are running across both clusters, even though the child clusters had zero GPU nodes before deployment.

Verify scale-in behavior

Scale down the Deployment to 2 replicas and re-apply:

kubectl apply -f deploy.yaml

After ten minutes, GPU node pools in the child clusters scale in to one node each. If you deleted the workload, the node count scales in to zero.

This confirms the full workflow: the fleet schedules workloads based on real-time inventory, scales out GPU nodes to run the workload, and automatically scales back in once the workload is removed—eliminating idle GPU costs.