ACK One Fleet: Scaling Agent Sandboxes from Single-Cluster to Multi-Cluster
By Jing Cai
As AI agents evolve from "conversational bots" to "action-oriented engines," every task-executing agent requires a secure, isolated, and ephemeral runtime environment: an Agent Sandbox.
However, as agent deployments scale in production, single-Kubernetes-cluster architectures quickly encounter fundamental bottlenecks:
- Capacity Bottlenecks: Sandboxes inherently consume compute, network, and storage resources. Every single cluster has an upper capacity limit.
- Concentrated Failure Domains: A single cluster constitutes a single failure domain. A failure in the Kubernetes or Sandbox control plane disrupts services for all users.
Conversely, when organizations attempt to build and manage multi-cluster architectures on their own, they often struggle with operational and scheduling inefficiencies:
- Lack of Global Watermark Visibility: Without real-time visibility into cluster capacity and load, new sandboxes must be assigned manually or via round-robin. This creates cluster hot-spotting, where some clusters are saturated while others sit idle.
- Inability to Route on Real-Time Capacity: Static routing rules cannot keep pace with dynamically fluctuating resource availability, frequently routing requests to near-saturated clusters and causing launch failures.
- No Cluster Health Awareness or Failover: Without automated health detection and circuit breaking, traffic continues to hit unhealthy clusters. Furthermore, single-cluster outages lack automated cross-cluster failover mechanisms.
- Low Utilization and High Ops Overhead: Inability to perform cross-cluster peak shaving forces teams to over-provision individual clusters to handle spikes, drastically driving up costs. Additionally, manual rebalancing during outages is error-prone and fails to scale.
To solve these challenges, ACK One Fleet delivers a dedicated multi-cluster Agent Sandbox architecture with clear objectives: expanding total sandbox scale, delivering high availability and failover, maximizing resource utilization and startup speed via multi-cluster scheduling, and evolving to support specialized AI agent workloads.
What is ACK One Multi-Cluster Agent Sandbox?
Agent Sandbox is a production-grade sandbox compute engine designed specifically for AI agents. It provides MicroVM-level runtime isolation, in-memory suspend/resume, and checkpoint cloning capabilities. It delivers extreme elasticity (scaling up to 15,000 sandboxes per minute), full compatibility with the Kubernetes-native ecosystem, and out-of-the-box integration with popular AI Agent frameworks such as the E2B SDK and AgentScope.
ACK One Fleet (Alibaba Cloud Distributed Cloud Container Platform for Kubernetes) provides a unified management control plane for multi-cluster and multi-region Kubernetes deployments. It governs self-managed, hybrid-cloud, and multi-region ACK clusters, enabling unified workload deployment, policy governance, and traffic management from a single pane of glass.
Building on this foundation, ACK One Fleet introduces specialized scheduling enhancements for Agent Sandboxes, elevating operations from single-cluster execution to multi-cluster collaboration. Users access the system through a unified Fleet endpoint with virtually zero architectural changes, interacting with the multi-cluster sandbox fabric just as seamlessly as they would with a single cluster.
Architecture & Key Capabilities: The Dual-Path Design (E2B SDK & K8s CR)

Users can connect to the sandbox via two standard paradigms—directly matching the dual-protocol API design of the sandbox foundation. ACK One Fleet seamlessly extends both paradigms across multiple clusters, while actual sandbox compute remains hosted within child member clusters.
Path A: E2B SDK — Global Control Plane with Decoupled Control & Data Planes
E2B is a widely adopted open-source code sandbox framework. ACK One multi-cluster Agent Sandbox natively supports the E2B ecosystem:
- Propagation: Users first deploy the SandboxSet across multiple member clusters via Fleet.
- Access: Requests initiated by the E2B SDK Client reach Fleet’s multi-cluster gateway.
- Global Scheduling: The Global Sandbox Scheduler handles sandbox creation requests by selecting the target cluster. Remaining lifecycle requests are resolved via the Global Route Table.
- Ingress Routing: The multi-cluster gateway routes traffic to the ALB Ingress of the target child cluster's sandbox manager.
- Execution: Each child cluster provides sandbox hosting via its local Ingress, sandbox gateway, manager, and controller.

Key Architectural Highlights
• Strict Separation of Control and Data Planes: Fleet proxies control-plane traffic only. It handles the sandbox lifecycle (creation, suspend, resume, etc.), while data-plane traffic terminates directly at the child clusters.
• Direct Child-Cluster Data Paths: In AI code sandbox scenarios, sandboxes frequently download large model weights, fetch massive datasets, or maintain high-throughput external API sessions. Routing data traffic through a centralized control plane creates severe network bottlenecks and introduces single points of failure. Decoupling ensures a lightweight, stable control plane while unlocking peak data-plane throughput.
Global Scheduling Capabilities
- Single-Cluster Capacity Limits: Enforces capacity thresholds per cluster during multi-cluster scheduling to prevent individual cluster overload.
- Watermark-Balanced Scheduling: Evenly distributes workloads across clusters based on real-time sandbox utilization watermarks.
- Multi-Cluster Failover (Manual & Automated): Redirects new sandbox creation requests to healthy clusters. Manual switching is ideal for routine maintenance: new sandbox requests bypass clusters undergoing maintenance without impacting existing running instances (e.g., active connections, code execution, or suspend/resume operations).
- Warm-Pool-Aware Affinity Scheduling: Prioritizes clusters with ample pre-warmed sandboxes to minimize startup latency and eliminate cold starts—an essential requirement for fast-paced Reinforcement Learning (RL) training loops.
Path B: Kubernetes CR (SandboxClaim) — Cloud-Native Continuity
For engineering teams deeply invested in the Kubernetes ecosystem, multi-cluster Agent Sandbox provides a declarative management path built on native Custom Resource Definitions (CRDs).
Workflow:
- Users deploy the SandboxSet to multiple clusters via Fleet.
- Users create a SandboxClaim alongside a PropagationPolicy on Fleet, dynamically claiming a sandbox instance from the scheduled cluster.

Requiring minimal configuration, this path inherits all capabilities of the E2B path while introducing enhanced global scheduling features:
- Single-cluster capacity control
- Watermark-balanced scheduling
- Multi-cluster failover
- Warm-pool-aware affinity scheduling
- Cluster Priority Scheduling: Enables cross-region tiered routing (e.g., prioritizing primary regions and spilling over to secondary regions only when primary cluster capacities are reached).
Summary
ACK One Fleet elevates the Agent Sandbox from single-cluster execution to multi-cluster collaboration: exposing a single endpoint upstream while governing multi-region, multi-account ACK clusters downstream. It preserves all native Agent Sandbox features—MicroVM isolation, in-memory suspend/resume, and checkpoint cloning—while breaking through single-cluster scale and resilience ceilings via dual E2B and K8s CR paths.
For enterprise teams, this delivers four major advantages with minimal friction: massive scale (workload absorption across multiple control planes), multi-region high availability (cross-cluster failover and smooth traffic draining), optimized resource utilization (enforced capacity limits with watermark balancing), and faster sandbox launches (elimination of cold-start latency for RL workflows).
Moving forward, ACK One Fleet will continue enhancing its multi-cluster scheduling engines to accommodate diverse, complex AI agent architectures—empowering production-grade agent workloads to flow autonomously across clusters.
Get Started
Explore the official documentation:
- Managing Agent Sandbox with ACK One multi-cluster Fleet
- ACK One Fleet overview
- Agent Sandbox user guide
Join the ACK One customer community on DingTalk to connect with our engineering team (DingTalk Group ID: 35688562).
Read previous post:
-
-
Alibaba Container Service
230 posts | 34 followers
Follow
Comments
-
-
Alibaba Container Service
230 posts | 34 followers
Follow
Related Products
-
Container Service for Kubernetes
Alibaba Cloud Container Service for Kubernetes is a fully managed cloud container management service that supports native Kubernetes and integrates with other Alibaba Cloud products.
Learn More -
Container Compute Service (ACS)
A cloud computing service that provides container compute resources that comply with the container specifications of Kubernetes
Learn More -
Application High Availability Service
Application High Available Service is a SaaS-based service that helps you improve the availability of your applications.
Learn More
More Posts by Alibaba Container Service
- Deep Dive: How Kimi's AI Agent Runs on Alibaba Cloud
- Caching is Efficiency: Achieving Precise LLM Cache Hits with Alibaba Cloud ACK GIE
- ACK One Fleet Multi-Cluster Canary Release: A "Safety Valve" for AI Inference Services
- Intelligent Scheduling for AI Inference: Cluster-Level Priority Elastic Scheduling
- When Agents Meet Workflows—Can Intelligence Become More Controllable?
- Koordinator Column 1: Viewing AI Computing Power's "Rigidity" and "Elasticity" through Gang Scheduling
- Container Technology Evolution for LLMs and AI Agents
- How ASM Ambient Mode Innovates Kubernetes Egress Traffic Management
- Best Practices for Ray on ACK: Secure Deployment of AI Data Processing/Training/Inference Environments