Community Blog ACK One Fleet: Scaling Agent Sandboxes from Single-Cluster to Multi-Cluster

ACK One Fleet: Scaling Agent Sandboxes from Single-Cluster to Multi-Cluster

This article introduces ACK One Fleet's multi-cluster scaling for AI agent sandboxes, optimizing global scheduling, availability, and resource utilization.

By Jing Cai

As AI agents evolve from "conversational bots" to "action-oriented engines," every task-executing agent requires a secure, isolated, and ephemeral runtime environment: an Agent Sandbox.

However, as agent deployments scale in production, single-Kubernetes-cluster architectures quickly encounter fundamental bottlenecks:

  1. Capacity Bottlenecks: Sandboxes inherently consume compute, network, and storage resources. Every single cluster has an upper capacity limit.
  2. Concentrated Failure Domains: A single cluster constitutes a single failure domain. A failure in the Kubernetes or Sandbox control plane disrupts services for all users.

Conversely, when organizations attempt to build and manage multi-cluster architectures on their own, they often struggle with operational and scheduling inefficiencies:

  1. Lack of Global Watermark Visibility: Without real-time visibility into cluster capacity and load, new sandboxes must be assigned manually or via round-robin. This creates cluster hot-spotting, where some clusters are saturated while others sit idle.
  2. Inability to Route on Real-Time Capacity: Static routing rules cannot keep pace with dynamically fluctuating resource availability, frequently routing requests to near-saturated clusters and causing launch failures.
  3. No Cluster Health Awareness or Failover: Without automated health detection and circuit breaking, traffic continues to hit unhealthy clusters. Furthermore, single-cluster outages lack automated cross-cluster failover mechanisms.
  4. Low Utilization and High Ops Overhead: Inability to perform cross-cluster peak shaving forces teams to over-provision individual clusters to handle spikes, drastically driving up costs. Additionally, manual rebalancing during outages is error-prone and fails to scale.

To solve these challenges, ACK One Fleet delivers a dedicated multi-cluster Agent Sandbox architecture with clear objectives: expanding total sandbox scale, delivering high availability and failover, maximizing resource utilization and startup speed via multi-cluster scheduling, and evolving to support specialized AI agent workloads.

What is ACK One Multi-Cluster Agent Sandbox?

Agent Sandbox is a production-grade sandbox compute engine designed specifically for AI agents. It provides MicroVM-level runtime isolation, in-memory suspend/resume, and checkpoint cloning capabilities. It delivers extreme elasticity (scaling up to 15,000 sandboxes per minute), full compatibility with the Kubernetes-native ecosystem, and out-of-the-box integration with popular AI Agent frameworks such as the E2B SDK and AgentScope.

ACK One Fleet (Alibaba Cloud Distributed Cloud Container Platform for Kubernetes) provides a unified management control plane for multi-cluster and multi-region Kubernetes deployments. It governs self-managed, hybrid-cloud, and multi-region ACK clusters, enabling unified workload deployment, policy governance, and traffic management from a single pane of glass.

Building on this foundation, ACK One Fleet introduces specialized scheduling enhancements for Agent Sandboxes, elevating operations from single-cluster execution to multi-cluster collaboration. Users access the system through a unified Fleet endpoint with virtually zero architectural changes, interacting with the multi-cluster sandbox fabric just as seamlessly as they would with a single cluster.

Architecture & Key Capabilities: The Dual-Path Design (E2B SDK & K8s CR)

1

Users can connect to the sandbox via two standard paradigms—directly matching the dual-protocol API design of the sandbox foundation. ACK One Fleet seamlessly extends both paradigms across multiple clusters, while actual sandbox compute remains hosted within child member clusters.

Path A: E2B SDK — Global Control Plane with Decoupled Control & Data Planes

E2B is a widely adopted open-source code sandbox framework. ACK One multi-cluster Agent Sandbox natively supports the E2B ecosystem:

  1. Propagation: Users first deploy the SandboxSet across multiple member clusters via Fleet.
  2. Access: Requests initiated by the E2B SDK Client reach Fleet’s multi-cluster gateway.
  3. Global Scheduling: The Global Sandbox Scheduler handles sandbox creation requests by selecting the target cluster. Remaining lifecycle requests are resolved via the Global Route Table.
  4. Ingress Routing: The multi-cluster gateway routes traffic to the ALB Ingress of the target child cluster's sandbox manager.
  5. Execution: Each child cluster provides sandbox hosting via its local Ingress, sandbox gateway, manager, and controller.

2

Key Architectural Highlights

• Strict Separation of Control and Data Planes: Fleet proxies control-plane traffic only. It handles the sandbox lifecycle (creation, suspend, resume, etc.), while data-plane traffic terminates directly at the child clusters.

• Direct Child-Cluster Data Paths: In AI code sandbox scenarios, sandboxes frequently download large model weights, fetch massive datasets, or maintain high-throughput external API sessions. Routing data traffic through a centralized control plane creates severe network bottlenecks and introduces single points of failure. Decoupling ensures a lightweight, stable control plane while unlocking peak data-plane throughput.

Global Scheduling Capabilities

  1. Single-Cluster Capacity Limits: Enforces capacity thresholds per cluster during multi-cluster scheduling to prevent individual cluster overload.
  2. Watermark-Balanced Scheduling: Evenly distributes workloads across clusters based on real-time sandbox utilization watermarks.
  3. Multi-Cluster Failover (Manual & Automated): Redirects new sandbox creation requests to healthy clusters. Manual switching is ideal for routine maintenance: new sandbox requests bypass clusters undergoing maintenance without impacting existing running instances (e.g., active connections, code execution, or suspend/resume operations).
  4. Warm-Pool-Aware Affinity Scheduling: Prioritizes clusters with ample pre-warmed sandboxes to minimize startup latency and eliminate cold starts—an essential requirement for fast-paced Reinforcement Learning (RL) training loops.

Path B: Kubernetes CR (SandboxClaim) — Cloud-Native Continuity

For engineering teams deeply invested in the Kubernetes ecosystem, multi-cluster Agent Sandbox provides a declarative management path built on native Custom Resource Definitions (CRDs).

Workflow:

  1. Users deploy the SandboxSet to multiple clusters via Fleet.
  2. Users create a SandboxClaim alongside a PropagationPolicy on Fleet, dynamically claiming a sandbox instance from the scheduled cluster.

3

Requiring minimal configuration, this path inherits all capabilities of the E2B path while introducing enhanced global scheduling features:

  1. Single-cluster capacity control
  2. Watermark-balanced scheduling
  3. Multi-cluster failover
  4. Warm-pool-aware affinity scheduling
  5. Cluster Priority Scheduling: Enables cross-region tiered routing (e.g., prioritizing primary regions and spilling over to secondary regions only when primary cluster capacities are reached).

Summary

ACK One Fleet elevates the Agent Sandbox from single-cluster execution to multi-cluster collaboration: exposing a single endpoint upstream while governing multi-region, multi-account ACK clusters downstream. It preserves all native Agent Sandbox features—MicroVM isolation, in-memory suspend/resume, and checkpoint cloning—while breaking through single-cluster scale and resilience ceilings via dual E2B and K8s CR paths.

For enterprise teams, this delivers four major advantages with minimal friction: massive scale (workload absorption across multiple control planes), multi-region high availability (cross-cluster failover and smooth traffic draining), optimized resource utilization (enforced capacity limits with watermark balancing), and faster sandbox launches (elimination of cold-start latency for RL workflows).

Moving forward, ACK One Fleet will continue enhancing its multi-cluster scheduling engines to accommodate diverse, complex AI agent architectures—empowering production-grade agent workloads to flow autonomously across clusters.

Get Started

Explore the official documentation:

Join the ACK One customer community on DingTalk to connect with our engineering team (DingTalk Group ID: 35688562).

0 0 0
Share on

Alibaba Container Service

230 posts | 34 followers

Comments