All Products
Search
Document Center

Container Compute Service:Create an agent sandbox in an ACK cluster

Last Updated:Apr 01, 2026

The customized SDK bypasses the native E2B SDK's wildcard domain requirement, simplifying deployment.

Prerequisites

  1. Create an ACK cluster.

  2. Upgrade the acs-virtual-node component to v2.17.0 or later.

  3. Upgrade the Kube Scheduler component.

    Cluster version

    Kube Scheduler component version

    v1.28

    v1.28.12-aliyun-1.4.6 or later

    v1.30

    v1.30.3-aliyun-1.6.2 or later

    v1.31

    v1.31.0-aliyun-1.5.2 or later

    v1.32

    v1.32.0-apsara.6.11.11.3187ac8f or later

  4. Upgrade the ack-agent-sandbox-controller component to v0.5.2 or later.

  5. Upgrade the ack-sandbox-manager component to v0.3.2 or later.

Install components

On the Clusters page, click the name of the target cluster. In the left navigation pane, choose Add-ons.

  1. Install an Ingress Controller component: Install any Ingress Controller component supported by ACK to access the sandbox-manager service from outside the cluster. This topic uses the ALB Ingress Controller as an example, which creates a public-facing ALB instance.

  2. Install the ack-agent-sandbox-controller component: Install the component with the default settings. The component version must be v0.5.2 or later.

  3. Install the ack-sandbox-manager component: After you complete the environment configuration, set className to alb (the IngressClass automatically created in Step 1), set domain to your actual domain name, and set adminApiKey to a custom API key. Leave the other settings at their default values. After installation, a route named sandbox-manager is created in the sandbox-system namespace.

    1. Demo environment configuration:

      1. Get the ALB DNS name: On the custom resources page, view the YAML of the AlbConfig resource object alb, or run the command kubectl get albconfig alb -o jsonpath='{.status.loadBalancer.dnsname}' to get the ALB DNS name, such as alb-*****62roo70i*****.cn-hangzhou.alb.aliyuncsslb.com. Set the domain parameter to this DNS name.

      2. Update the sandbox-manager route: In the sandbox-system namespace, click Update next to the sandbox-manager route. Temporarily disable the TLS configuration. After the update is complete, wait about one minute and refresh the route page. The endpoint information for the route appears.

    2. Production environment configuration: For detailed instructions on preparing a domain name, configuring DNS resolution, and requesting a certificate, see Going Live. If you use the ALB Ingress Controller, you must also add an HTTPS:443 listener for the ALB instance and Ingress.

    Parameter details

    Parameter

    Parameter

    Description

    sandboxManager

    replicaCount

    The number of sandbox-manager instances. The default value is 3.

    E2B

    domain

    The domain name for E2B. This is the domain name that you prepared in Step 3.

    Enable E2B_API_KEY authentication

    Specifies whether to enable API key authentication. This feature is enabled by default.

    adminApiKey

    When authentication is enabled, this parameter specifies the initial key during the first installation. Replace the default value with your custom API key.

    Controller

    logLevel

    The log level for the controller. The default value is 1.

    resources.requests.cpu

    The CPU resource request for the controller. The default value is 2.

    resources.requests.memory

    The memory resource request for the controller. The default value is 4Gi.

    Proxy

    resources.requests.cpu

    The CPU resource request for the proxy. The default value is 2.

    resources.requests.memory

    The memory resource request for the proxy. The default value is 4Gi.

    Ingress

    className

    The name of the IngressClass configured in the cluster, such as alb or mse.

Create a sandbox environment

E2B SDK-compatible method

In the following steps, you first create and warm up an E2B template by using a SandboxSet, and then create a sandbox environment by using the customized ACK E2B Python SDK.

Step 1: Create and warm up an E2B template

  1. In the navigation pane on the left, choose Workloads > custom resources. Select the CustomResourceDefinition tab, and then click Create from YAML.

  2. Use the following YAML to create a SandboxSet resource. This registers an E2B template named code-interpreter with ack-sandbox-manager. The template is based on the official E2B code-interpreter image and is used to execute Python code.

    apiVersion: agents.kruise.io/v1alpha1
    kind: SandboxSet
    metadata:
      name: code-interpreter
      namespace: default
    spec:
      scaleStrategy:
        maxUnavailable: 500
      replicas: 10  # The size of the warm pool. We recommend setting this slightly larger than the anticipated burst of requests.
      template:
        # metadata:
          # annotations:
            # Optional. If you leave this empty, the security group configuration in acs-profile is used.
            # Ensure that the corresponding security group has enough available IP addresses, at least more than the number of replicas.
            # Insufficient IPs can slow down or even fail sandbox creation.
            # network.alibabacloud.com/security-group-ids: sg-8******
          # labels:
            # Optional. Used to schedule sandbox pods to ACS in an ACK cluster.
            # alibabacloud.com/acs: "true"
        spec:
          initContainers:
            # Declare agent-runtime as a native sidecar to automatically inject runtime components like envd into the sandbox container.
            - name: runtime
              image: registry-cn-hangzhou-vpc.ack.aliyuncs.com/acs/agent-runtime:v0.0.5
              command: [ "sh", "/workspace/entrypoint_inner.sh" ]
              volumeMounts:
                # Shared directory with the main container.
                - name: envd-volume
                  mountPath: /mnt/envd
              env:
                - name: ENVD_DIR
                  value: /mnt/envd
                # This environment variable allows the sidecar to share the resources of the main container without incurring extra costs.
                - name: __IGNORE_RESOURCE__
                  value: "true"
              restartPolicy: Always
          containers:
          - name: sandbox
            # The official e2b code-interpreter image. It supports pulling from all regions and VPCs.
            image: registry-cn-hangzhou-vpc.ack.aliyuncs.com/acs/code-interpreter:v1.6
            imagePullPolicy: IfNotPresent
            # We recommend that you set resource requests. Otherwise, in an ACK environment, the sandbox may be assigned very small specifications, which can affect performance.
            resources:
              limits:
                cpu: 1
                memory: 1Gi
              requests:
                cpu: 1
                memory: 1Gi
            startupProbe:
              failureThreshold: 10
              httpGet:
                path: /health
                port: 49999
              initialDelaySeconds: 1
              periodSeconds: 2
              timeoutSeconds: 1
            env:
              # Specifies the location of the envd component injected by the runtime.
              - name: ENVD_DIR
                value: /mnt/envd
            volumeMounts:
              # Shared directory with the runtime.
              - name: envd-volume
                mountPath: /mnt/envd
            # Starts the envd service by using a postStart hook.
            lifecycle:
              postStart:
                exec:
                  command: [ "/bin/bash", "-c", "/mnt/envd/envd-run.sh" ]
          # Ensures quick container termination to increase the probability of reuse.
          terminationGracePeriodSeconds: 1
          volumes:
            - name: envd-volume
              emptyDir: { }

    Notes on using the code-interpreter image

    • Image pull: The registry-cn-hangzhou-vpc.ack.aliyuncs.com/acs/code-interpreter image is an example image created by ACK based on the e2b-code-interpreter project. It is compatible with the run_code API of the E2B client and is recommended for direct use. You can change the region in the image address to your actual region to speed up the pull, or remove "-vpc" to pull over the public internet.

    • Use a custom image: If you need to use E2B code-interpreter (either directly or as a base image), we recommend using the versioned images provided by ACK. ACK does not guarantee runtime compatibility with the official E2B latest image. If you use a custom image, ensure that it meets the following conditions:

      1. It includes basic commands such as cp, mv, and mkdir.

      2. It includes bash, with the executable located at /bin/bash.

      When you use a custom image, the E2B run_code method is temporarily unavailable.

  3. In the navigation pane on the left, click Pods. Select the default namespace to view the created and warmed-up code-interpreter sandbox pods.

Step 2: Create a sandbox environment

  1. Install Git and Python in your local environment.

  2. Install the E2B Python SDK.

    pip install e2b-code-interpreter==2.4.1
  3. Configure environment variables.

    # Use the default domain from the ack-sandbox-manager component installation. Do not include "*". Modify as needed.
    export E2B_DOMAIN=your.domain.com
    # Use the default API Key from the ack-sandbox-manager component installation. Modify as needed.
    export E2B_API_KEY=admin-987654321
  4. Run the git clone https://github.com/openkruise/agents.git command to download the customized ACK SDK to your local machine.

  5. Navigate to the /agents/sdk/customized_e2b directory of the customized SDK and save the following code as a main.py file.

    # Import the E2B SDK
    from e2b_code_interpreter import Sandbox
    from kruise_agents.patch_e2b import patch_e2b
    
    patch_e2b(https=False)  # patch sdk
    
    sbx: Sandbox = Sandbox.create(template="code-interpreter")
    print(f"sandbox id: {sbx.sandbox_id}")
    result = sbx.run_code("print('hello, world')")
    print(f"run code result: {result}")
    text = input("enter some text to be saved to file 'text.txt' inside sandbox:  ")
    sbx.files.write("text.txt", text)
    print(f"read file from sandbox via files api: [{sbx.files.read('text.txt')}]")
    print(f"read file from sandbox via commands api: [{sbx.commands.run('cat text.txt')}]")
    input("press ENTER to kill the sandbox")
    print(sbx.kill())
    

    Create API: metadata parameter

    For usage examples of the metadata parameter in the create API, see Usage examples.

    Key

    Description

    Value

    e2b.agents.kruise.io/never-timeout

    Specifies whether to disable automatic cleanup for the sandbox. If set to true, this sandbox is not automatically cleaned up based on a timeout.

    • false

    • true

    e2b.agents.kruise.io/image

    Specifies the runtime image for the requested sandbox. If the image is different from the one declared in the warm pool, the image is upgraded in-place.

    A specific image name, for example, ghcr.io/openclaw/openclaw:****

    e2b.agents.kruise.io/wait-ready-timeout-seconds

    Mainly used for scenarios where an image is updated upon request. Sets the maximum time to wait for the image update to complete.

    Integer

    e2b.agents.kruise.io/csi-volume-config

    Supports dynamically mounting NAS or OSS when you request a sandbox. The format is a JSON string that supports fine-grained configuration for multiple mount points. Each mount point supports the following parameters:

    1. pvName: The name of the PersistentVolume (PV) for the OSS/NAS volume.

    2. mountPath: The mount directory inside the sandbox.

    3. subPath: The BucketPath for OSS/NAS, which represents an absolute path.

    volume_config = [

    {"pvName": "oss-pv-sandbox-system", "mountPath": "/home/data-oss1", "subPath": "data-subPath1","readOnly": true}]

    e2b.agents.kruise.io/claim-timeout-seconds

    The maximum time to wait to acquire an available sandbox. If this time is exceeded, the request fails.

    Integer. The default value is 60.

    e2b.agents.kruise.io/create-on-no-stock

    By default, sandboxes are always acquired from the warm pool. If no sandbox is available, the request waits for one. This setting allows a new sandbox to be created directly from the template to reduce wait time.

    • false

    • true

    Default: false

    e2b.agents.kruise.io/skip-init-runtime

    Specifies whether to initialize the agent-runtime in the sandbox instance when it is requested. Set this to false if there is no runtime process or if high customization is needed.

    • false

    • true

    Default: true

    e2b.agents.kruise.io/reserve-failed-sandbox

    When an error occurs while acquiring a sandbox, the management component immediately cleans up the failed sandbox. This field lets the component retain the failed sandbox instance for troubleshooting.

    • false

    • true

    Default: false

    Custom metadata

    When you acquire a sandbox, you can use custom key-value pairs to add annotations to the sandbox instance.

    A custom value, for example, {"userId": "alice"}.

  6. Run the main.py file to create and verify the sandbox environment.

    After the prompt appears for the first time, enter text such as acs agent sandbox and press Enter. This action writes acs agent sandbox to the /home/user/text.txt file in the Pod named code-interpreter-29***. If you press Enter again, the current sandbox environment is deleted.
    python main.py

    Expected output:

    sandbox id: default--code-interpreter-29***
    run code result: Execution(Results: [], Logs: Logs(stdout: ['hello, world\n'], stderr: []), Error: None)
    enter some text to be saved to file 'text.txt' inside sandbox:  ack agent sandbox
    read file from sandbox via files api: [ack agent sandbox]
    read file from sandbox via commands api: [CommandResult(stderr='', stdout='ack agent sandbox', exit_code=0, error='')]
    press ENTER to kill the sandbox
    True

Sandbox CR method

  1. In the navigation pane on the left, choose Workloads > custom resources. Select the CustomResourceDefinition tab, and then click Create from YAML.

  2. Use the following YAML to create a Sandbox custom resource object. This object is based on the official E2B code-interpreter image and executes Python code.

    apiVersion: agents.kruise.io/v1alpha1
    kind: Sandbox
    metadata:
      name: code-interpreter
    spec:
      template:
        metadata:
          labels:
            agent: code-interpreter
        spec:
          initContainers:
            # Declare agent-runtime as a native sidecar to automatically inject runtime components like envd into the sandbox container.
            - name: runtime
              image: registry-cn-hangzhou-vpc.ack.aliyuncs.com/acs/agent-runtime:v0.0.5
              command: [ "sh", "/workspace/entrypoint_inner.sh" ]
              volumeMounts:
                # Shared directory with the main container.
                - name: envd-volume
                  mountPath: /mnt/envd
              env:
                - name: ENVD_DIR
                  value: /mnt/envd
                # This environment variable allows the sidecar to share the resources of the main container without incurring extra costs.
                - name: __IGNORE_RESOURCE__
                  value: "true"
              restartPolicy: Always
          containers:
          - name: sandbox
            image: registry-cn-hangzhou-vpc.ack.aliyuncs.com/acs/code-interpreter:v1.6
            imagePullPolicy: IfNotPresent
            resources:
              requests:
                cpu: 1
                memory: 1Gi
                ephemeral-storage: 30Gi
            startupProbe:
              failureThreshold: 10
              httpGet:
                path: /health
                port: 49999
              initialDelaySeconds: 1
              periodSeconds: 2
              timeoutSeconds: 1
            env:
              # Specifies the location of the envd component injected by the runtime.
              - name: ENVD_DIR
                value: /mnt/envd
            volumeMounts:
              # Shared directory with the runtime.
              - name: envd-volume
                mountPath: /mnt/envd
            # Starts the envd service by using a postStart hook.
            lifecycle:
              postStart:
                exec:
                  command: [ "/bin/bash", "-c", "/mnt/envd/envd-run.sh" ]
          # Ensures quick container termination to increase the probability of reuse.
          terminationGracePeriodSeconds: 1
          volumes:
            - name: envd-volume
              emptyDir: { }

    Notes on using the code-interpreter image

    • Image pull: The registry-cn-hangzhou-vpc.ack.aliyuncs.com/acs/code-interpreter image is an example image created by ACK based on the e2b-code-interpreter project. It is compatible with the run_code API of the E2B client and is recommended for direct use. You can change the region in the image address to your actual region to speed up the pull, or remove "-vpc" to pull over the public internet.

    • Use a custom image: If you need to use E2B code-interpreter (either directly or as a base image), we recommend using the versioned images provided by ACK. ACK does not guarantee runtime compatibility with the official E2B latest image. If you use a custom image, ensure that it meets the following conditions:

      1. It includes basic commands such as cp, mv, and mkdir.

      2. It includes bash, with the executable located at /bin/bash.

      When you use a custom image, the E2B run_code method is temporarily unavailable.

  3. In the navigation pane on the left, click Pods. Select the default namespace to view the created code-interpreter sandbox environment.

Going live

Prepare a domain name

The customized ACK SDK supports non-wildcard domains. Refer to Add or delete a private zone to configure the domains your.domain.com and *.your.domain.com and resolve them to the Ingress address. If access is entirely within the ACK cluster, you can also directly use the in-cluster Headless Service address: sandbox-manager.sandbox-system.svc.cluster.local.

Configure DNS resolution

Run the following command to view the access point information:

kubectl get ingress sandbox-manager -o jsonpath='{range .status.loadBalancer.ingress[*]}{.hostname}{.ip}{"\n"}{end}' -n sandbox-system

Based on the output, configure the DNS resolution for your.domain.com or *.your.domain.com. For more information about DNS resolution, see Quick start.

  • If the output is an IP address (for example, 47.114.***.***), create an A record for the host *.your.domain.com that points to this IP address.

  • If the output is a domain name (for example, alb-*****62roo70i*****.cn-hangzhou.alb.aliyuncsslb.com), create a CNAME record for the host *.your.domain.com that points to this domain name.

  • If multiple access points are returned, you can resolve the record to any one of them or configure round-robin DNS for all of them.

Request a certificate

The E2B client can send requests to the backend over HTTPS. For production scenarios, we recommend using one of the following methods for certificate provisioning.

(Recommended) Method 1: Use cert-manager

The following steps provide a best practice for managing and deploying a self-signed certificate for sandbox-manager by using cert-manager. Ensure that you have the kubectl CLI and permissions to operate cert-manager APIs.

Step 1: Install cert-manager

Refer to the official documentation to install cert-manager v1.14.

Because the latest version uses the community image repository (quay.io), we recommend using v1.14.
Step 2: Automate certificate management with cert-manager
  1. In the cert-manager.yaml file, replace *.your.domain.com and your.domain.com with your domain names.

    The example YAML uses a self-signed CA certificate. You can also use your own CA certificate.
  2. Run the command kubectl apply -f cert-manager.yaml to apply the configuration to your ACK cluster.

Step 3: Verify the certificate status
  1. Check if the certificate was created and issued correctly.

    kubectl get certificates -n sandbox-system
    kubectl describe certificate sandbox-manager-ingress-cert -n sandbox-system
    kubectl describe secret sandbox-manager-tls -n sandbox-system
  2. Check the Ingress status.

    kubectl get ingress sandbox-manager -n sandbox-system
    kubectl describe ingress sandbox-manager -n sandbox-system
Step 4: Configure client trust

If you are using a self-signed certificate, the client must trust the root CA certificate.

  1. Get the CA certificate.

    kubectl get secret sandbox-ca-key-pair -n sandbox-system -o jsonpath='{.data.tls\.crt}' | base64 -d > ca.crt
  2. On the client, set the SSL_CERT_FILE environment variable to the path of the retrieved CA certificate, or add the CA certificate to the system's trust store.

    export SSL_CERT_FILE=/path/to/ca.crt

Method 2: Self-signed certificates

Step 1: Create the certificate
  1. Create a self-signed certificate by using the generate-certificates.sh script. You can view the script's usage with the following command.

    bash generate-certificates.sh --help

    Expected output:

    Usage: generate-certificates.sh [OPTIONS]
    
    Options:
      -d, --domain DOMAIN     Specify certificate domain (default: example.com)
      -o, --output DIR        Specify output directory (default: .)
      -D, --days DAYS         Specify certificate validity days (default: 365)
      -h, --help              Show this help message
    
    Examples:
      generate-certificates.sh -d myapp.example.com
      generate-certificates.sh --domain api.example.com --days 730
  2. Run the command generate-certificates.sh -d your.domain.com to generate the certificate. After generation, the following files are created:

    • fullchain.pem: Server certificate public key

    • privkey.pem: Server certificate private key

    • ca-fullchain.pem: CA certificate public key

    • ca-privkey.pem: CA certificate private key

    This script generates certificates for both a specific domain (your.domain.com) and a wildcard domain (*.your.domain.com), ensuring compatibility with both the native E2B protocol and the customized OpenKruise E2B protocol.

Step 2: Install the certificate

Run the following command to create a TLS secret for the Ingress from your server certificate:

kubectl create secret tls sandbox-manager-tls \
        --cert=fullchain.pem \
        --key=privkey.pem -n sandbox-system
There might be a delay before the certificate takes effect. The exact timing depends on the Ingress controller you are using.
Step 3: Configure client trust

The client needs to set the SSL_CERT_FILE environment variable to the path of the CA public key file (ca-fullchain.pem) generated in Step 1:

export SSL_CERT_FILE=/path/to/ca-fullchain.pem
python main.py # Call the E2B SDK via Python

Method 3: Official certificates

Step 1: Prepare the certificate

The E2B client must send requests to the backend over HTTPS. For production scenarios, we recommend requesting an official domain certificate.

The following example uses Let's Encrypt to request a free test certificate. Let's Encrypt certificates require a public domain name.

  1. Install certbot by using your system's package manager (such as brew or snap). For more installation information, see the official documentation.

  2. Modify the -d and --email parameters to request a certificate for your domain your.domain.com and its wildcard subdomain. Follow the prompts in the command output to complete the verification process.

Because the issued certificate includes both a wildcard and a specific domain, you must perform the TXT record verification process twice consecutively.
sudo certbot certonly \
  --manual \
  --preferred-challenges=dns \
  --email your-email@example.com \
  --server https://acme-v02.api.letsencrypt.org/directory \
  --agree-tos \
  -d "your.domain.com" \
  -d "*.your.domain.com"
Step 2: Export the certificate
sudo cp /etc/letsencrypt/live/your.domain.com/fullchain.pem ./fullchain.pem
sudo cp /etc/letsencrypt/live/your.domain.com/privkey.pem ./privkey.pem
Step 3: Install the certificate
kubectl create secret tls sandbox-manager-tls \
  --cert=fullchain.pem \
  --key=privkey.pem \
  --namespace=sandbox-system