All Products
Search
Document Center

Container Service for Kubernetes:Use Sandbox resources to manage multi-cluster Agent Sandbox

Last Updated:Oct 20, 2026

AI applications can run Agent Sandbox across multiple associated clusters of an ACK One fleet. Configure the Sandbox capacity limit and E2B domain for each associated cluster to centrally schedule and manage Sandboxes across the clusters.

Solution overview

The ACK One fleet supports unified management and scheduling of Agent Sandbox across multiple clusters in three ways: the E2B SDK, Kubernetes SandboxClaim resources, and Sandbox resources. For information about how to use the E2B SDK and SandboxClaim resources, see Manage Agent Sandbox by using ACK One multi-cluster fleet.

This tutorial uses Sandbox resources. After a Sandbox is created in the fleet, a distribution policy (PropagationPolicy) matches the Sandbox resources in the namespace and distributes them to the associated clusters. The Sandbox capacity limit and the E2B domain of each associated cluster are declared in the sandbox-scheduler-config ConfigMap in the sandbox-system namespace of the fleet. After you configure the ConfigMap, the E2B domain is recorded in the sandbox.one.alibabacloud.com/e2b-domain annotation on the Sandbox resource.

Prerequisites

  1. Enable fleet management.

  2. Obtain the kubeconfig of the fleet in the ACK One console and connect to the fleet by using kubectl.

  3. Add at least two associated clusters to the fleet. For more information, see Manage associated clusters.

  4. Install the Agent Sandbox-related add-ons in all associated clusters. For more information, see Install add-ons.

    Important

    The version of the ack-agent-sandbox-controller add-on must be v0.6.6-release.1 or later.

  5. Install the AMC command-line tool. For more information, see AMC command-line help.

Preparations

Configure multi-cluster scheduling for the fleet:

  1. In the fleet, edit the sandbox-scheduler-config ConfigMap:

    kubectl edit cm sandbox-scheduler-config -n sandbox-system
  2. Configure the ConfigMap based on the following example. Replace the cluster IDs and the E2B domains with the actual values:

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: sandbox-scheduler-config
      namespace: sandbox-system
    data:
      clusters: |
        - name: <your-ack-cluster-1-id>
          maxSandboxes: 150000
          e2bDomain: <your-cluster-1-sandbox-e2b-domain>
        - name: <your-ack-cluster-2-id>
          maxSandboxes: 150000
          e2bDomain: <your-cluster-2-sandbox-e2b-domain>
      defaultMaxSandboxes: "100000"

    The following table describes the fields:

    Field

    Sub-field

    Required

    Description

    defaultMaxSandboxes

    -

    No

    The expected maximum number of Sandboxes in a single cluster, excluding the Sandbox replicas of a SandboxSet. If maxSandboxes is not configured in clusters, this field determines the Sandbox capacity limit of a single cluster. Default value: 100000. defaultMaxSandboxes

    clusters

    name

    Yes

    The cluster ID, such as c6cb44842ae2242e6bc5b96cea4f3xxxx.

    clusters

    maxSandboxes

    No

    The expected maximum number of Sandboxes in the cluster. If this field is left blank, the value of defaultMaxSandboxes takes effect.

    clusters

    e2bDomain

    No

    The E2B domain configured when Sandbox Manager was installed in the cluster, such as cluster1.example.com. After you configure this field, its value is recorded in the sandbox.one.alibabacloud.com/e2b-domain annotation on the Sandbox resource. In Step 3, retrieve the E2B domain from this annotation.

Step 1: Create a distribution policy in the fleet

  1. Create the namespace for the Sandboxes in the fleet. Make sure that the same namespace exists in all associated clusters.

  2. Create the following distribution policy (PropagationPolicy) in the namespace. The policy matches all Sandbox resources in the namespace and distributes them to the associated clusters specified by clusterAffinity. To select resources by label or by name, see Distribution policies and differentiation policies.

    Replace the namespace and the cluster IDs in the following example with the actual values:

    apiVersion: policy.one.alibabacloud.com/v1alpha1
    kind: PropagationPolicy
    metadata:
      name: sandbox-pp
      namespace: <your-namespace>
    spec:
      # false: when the resource on the fleet is deleted, the resources in the associated clusters are also deleted
      preserveResourcesOnDeletion: false
      resourceSelectors:
      - apiVersion: agents.kruise.io/v1alpha1
        kind: Sandbox
      placement:
        replicaScheduling:
          customSchedulingType: Gang
          replicaSchedulingType: Divided
        clusterAffinity:
          clusterNames:
          - <your-ack-cluster-1-id>
          - <your-ack-cluster-2-id>

    When preserveResourcesOnDeletion is set to false, deleting the Sandbox resource on the fleet also deletes the resources in the associated clusters.

Step 2: Create a Sandbox in the fleet

Create the Sandbox resource in the fleet. The following example creates a code-interpreter Sandbox. Replace the namespace with the namespace created in Step 1:

apiVersion: agents.kruise.io/v1alpha1
kind: Sandbox
metadata:
  generateName: code-interpreter-
  namespace: <your-namespace>
  labels:
    type: divided
spec:
  persistentContents:
  - filesystem
  runtimes:
  - name: csi
  - name: agent-runtime
  template:
    metadata:
      annotations:
        scaling.alibabacloud.com/enable-inplace-resource-resize: "true"
      labels:
        agents.kruise.io/sandbox-claimed: "false"
        agents.kruise.io/sandbox-pool: code-interpreter
        agents.kruise.io/sandbox-template: code-interpreter
        agents.kruise.io/template-hash: 846b7ff5d
        alibabacloud.com/acs: "true"
        alibabacloud.com/compute-class: agent-sandbox
        alibabacloud.com/compute-qos: default
    spec:
      automountServiceAccountToken: false
      containers:
      - image: registry-cn-zhangjiakou.ack.aliyuncs.com/acs/code-interpreter:v1.6
        imagePullPolicy: IfNotPresent
        name: sandbox
        resources:
          limits:
            cpu: "1"
            memory: 2Gi
          requests:
            cpu: "1"
            memory: 2Gi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30

Step 3: View Sandboxes and obtain the E2B domain

View the Sandbox distribution in the fleet and the associated clusters to verify the multi-cluster scheduling result of the previous steps.

  1. View the Sandboxes in the fleet:

    kubectl get sandbox -n <your-namespace>
  2. View the Sandboxes in the associated clusters. The -M flag queries all associated clusters. The -m flag queries one specified associated cluster. <your-ack-cluster-id> is the clusters.name value configured in Preparations:

    # View the Sandboxes in all associated clusters
    kubectl amc get sandbox -n <your-namespace> -M
    
    # View the Sandboxes in a specified associated cluster
    kubectl amc get sandbox -n <your-namespace> -m <your-ack-cluster-id>
  3. Obtain the E2B domain of a Sandbox in the fleet or an associated cluster. Replace <your-sandbox> with the name of the Sandbox:

    kubectl get sandbox <your-sandbox> -n <your-namespace> -ojsonpath='{.metadata.annotations.sandbox\.one\.alibabacloud\.com\/e2b-domain}'

Next steps

To manage Agent Sandbox in the fleet by using the E2B SDK or Kubernetes SandboxClaim resources, see Manage Agent Sandbox by using ACK One multi-cluster fleet.