Function Compute:Use Computer Use Sandbox

If a task only requires accessing web pages, clicking, filling forms, taking screenshots, or scraping dynamic pages, use Browser Use Sandbox and control the browser precisely through CDP. Use Computer Use Sandbox only when a task must operate desktop applications outside the browser or drive an interface visually through pixel coordinates and screenshots.

Use Cases

Comparison with Browser Use Sandbox

Prerequisites

1. Build a template such as my-desktop-template by following the Desktop Template example.

2. Configure E2B_API_KEY, E2B_API_URL, and E2B_DOMAIN for the same region as the template (read automatically by the SDK).

3. Install the Desktop SDK for your language; see Desktop Template example for the install commands and dependency versions.

Python Example

The following example creates a desktop sandbox, launches Chrome, takes a screenshot, performs mouse and keyboard actions, and starts an authenticated live stream.

"""Computer Use Sandbox example."""

import os

from dotenv import load_dotenv
from e2b_desktop import Sandbox

load_dotenv()

TEMPLATE = os.environ.get("E2B_DESKTOP_TEMPLATE", "my-desktop-template")

desktop = None
try:
    # timeout is in seconds and must cover desktop startup and the entire task.
    desktop = Sandbox.create(template=TEMPLATE, timeout=600)

    # Launch Chrome and wait for the window to render. wait is in milliseconds.
    desktop.launch("google-chrome", "https://example.com")
    desktop.wait(10000)

    # Mouse actions.
    desktop.move_mouse(640, 400)
    desktop.left_click()
    desktop.double_click()
    desktop.scroll("down", 5)

    # Type in a terminal to verify keyboard input and shortcuts.
    desktop.launch("xfce4-terminal")
    desktop.wait(3000)
    desktop.write("computer use")
    desktop.press(["ctrl", "a"])
    desktop.write("echo ")
    desktop.press("enter")
    desktop.wait(1000)

    # screenshot returns PNG bytes for a vision model or local file.
    image = desktop.screenshot()
    with open("desktop.png", "wb") as f:
        f.write(image)

    # Start a live stream protected by a VNC password.
    desktop.stream.start(require_auth=True)
    auth_key = desktop.stream.get_auth_key()
    stream_url = desktop.stream.get_url(auth_key=auth_key)
    # Pass stream_url to a controlled backend. Do not write it to public logs.

    # Run the Agent's screenshot -> decision -> action loop here.

    desktop.stream.stop()
finally:
    if desktop is not None:
        desktop.kill()

TypeScript Example

Save the following code as computer-use.ts:

import "dotenv/config";
import { writeFileSync } from "node:fs";
import { Sandbox } from "@e2b/desktop";

const TEMPLATE = process.env.E2B_DESKTOP_TEMPLATE ?? "my-desktop-template";

let desktop: Sandbox | undefined;
try {
  // timeoutMs is in milliseconds and must cover desktop startup and the entire task.
  desktop = await Sandbox.create(TEMPLATE, { timeoutMs: 600_000 });

  await desktop.launch("google-chrome", "https://example.com");
  // Wait for the application window to render before taking a screenshot.
  await desktop.wait(10_000);

  await desktop.moveMouse(640, 400);
  await desktop.leftClick();
  await desktop.doubleClick();
  await desktop.scroll("down", 5);

  await desktop.launch("xfce4-terminal");
  await desktop.wait(3_000);
  await desktop.write("computer use");
  await desktop.press(["ctrl", "a"]);
  await desktop.write("echo ");
  await desktop.press("enter");
  await desktop.wait(1_000);

  const image = await desktop.screenshot();
  writeFileSync("desktop.png", image);

  await desktop.stream.start({ requireAuth: true });
  const authKey = desktop.stream.getAuthKey();
  const streamUrl = desktop.stream.getUrl({ authKey });
  // Pass streamUrl to a controlled backend. Do not write it to public logs.

  // Run the Agent's screenshot -> decision -> action loop here.

  await desktop.stream.stop();
} finally {
  await desktop?.kill();
}

Run the example:

npm install
npx tsx computer-use.ts

For Desktop template capabilities and a Python-to-TypeScript API reference, see Desktop Template.

Live Streaming and Authentication

• stream.start(require_auth=True) / stream.start({ requireAuth: true }) generates a password for the VNC stream.

• You must call get_auth_key() / getAuthKey() only after enabling require_auth / requireAuth. Otherwise, the SDK throws an error.

• stream.get_url() returns a noVNC URL. For read-only observation, use view_only=True in Python or viewOnly: true in TypeScript.

• Only one stream can run at a time. Version v0.0.44 supports whole-desktop streaming but does not support streaming an individual window.

Accessing /vnc.html and /websockify through the Function Compute public gateway also requires the sandbox X-Access-Token. The auth_key generated by require_auth is the VNC stream password. These are two independent authentication layers. Because standard browsers cannot add arbitrary request headers to pages and WebSocket connections, expose live viewing to end users through a controlled reverse proxy that injects X-Access-Token. Do not expose the sandbox token to the frontend or public logs.

Security and Production Recommendations

• Credential isolation: Inject model keys, account passwords, and business tokens through runtime environment variables. Do not write them into the template image.

• Stream access control: Enable require_auth for live streams, use view_only for read-only demonstrations, and do not write URLs containing passwords to public logs.

• Prompt injection protection: Do not let an Agent treat on-screen text as system instructions without validation. Require human approval for sensitive actions such as submitting, deleting, or making payments.

• Resources and lifecycle: Set a reasonable task timeout and always call kill() in finally.

• Action auditing: Record the necessary screenshots, mouse coordinates, key sequences, and target windows for reproduction and tracing.

• Least privilege: Restrict the domains, applications, and writable directories available to the sandbox.

Related Documentation

• Desktop Template

• Browser Use Sandbox

• AIO Sandbox

• Upload and Download Files