How to use AgentSight - Alibaba Cloud Linux - Alibaba Cloud Documentation Center

AgentSight is an eBPF-based observability tool for AI agents. It provides fine-grained data collection and correlation analysis across the entire agent lifecycle, without affecting your business logic.

Using AgentSight

Product overview

Core capabilities

AgentSight provides the following capabilities:

Token consumption analysis: Measures and attributes token consumption during agent execution. You can query data flexibly by time range or the last N hours, with automatic period-over-period comparisons. The analysis breaks down consumption by multiple dimensions, such as agent, task, or role, with a granularity down to a single LLM call.
Behavior audit: Provides end-to-end recording and tracing of an agent's LLM calls and process execution. It captures key metadata for every LLM call—such as the provider and model version—along with the command-line arguments of processes. The system supports flexible, multi-dimensional filtering by time, process ID, and event type, and offers visualized summary statistics for analysis.
Dashboard visualization: The web-based interface provides an intuitive view of token consumption, agent status, and session details. You can view real-time data updates in your browser. The dashboard can be deployed on a remote server and accessed directly from your local browser without logging into the server. With the dashboard, you can view token consumption trends over time, monitor agent process status in real time and restart abnormal processes, and drill down into the complete trace for each session. This includes user inputs, model prompts, the thinking process, and the token consumption distribution for each step, helping you analyze model call efficiency and optimize costs.

Scope

This tool supports OpenClaw and Copilot Shell (in non-AK/SK authentication scenarios).

Installation

For details, see Quick Start.

Conversational interaction

AgentSight provides a conversational skill that can be installed in various AI agents. You can use natural language to perform operations instead of memorizing CLI commands:

Check token consumption: For example, "How many tokens have I used today?"
Query audit logs: For example, "Show me today's LLM call records."

If you are using Copilot Shell (cosh), this skill is built-in. You can use the natural language commands above directly. The system automatically calls AgentSight to perform the query and return the analysis.

CLI commands

agentsight trace — Start eBPF tracing

Note: This service is started by default and does not need to be run manually.

Starts eBPF-based tracking of AI agent activity.

agentsight trace # root privileges required

agentsight serve — Start API and dashboard

Note: This service is started by default and binds to 0.0.0.0:7396. You do not need to run it manually.

Starts the HTTP API server, which provides an embedded dashboard UI.

agentsight serve --host 0.0.0.0 --port 7396 # root privileges required

This command binds to all network interfaces. You can access it at the server's public IP address: http://<your-server-public-ip>:7396

Ensure that your server's firewall or security group allows inbound traffic on port 7396.

agentsight token — Query token usage

Queries token usage data.

# View today's usage
agentsight token

agentsight audit — Query audit events

Queries audit events, such as LLM calls and process operations.

# View recent events
agentsight audit
# Filter by PID and type
agentsight audit --pid 12345 --type llm
# Get a summary
agentsight audit --summary

agentsight discover — Scan for agents

Discovers running AI agents on the system.

# Scan for agents
agentsight discover
# List known types
agentsight discover --list-known

Dashboard

The dashboard is the web UI for AgentSight, used to view conversation history, trace details, and token statistics.

Dashboard features

The dashboard provides the following core features:

Token consumption overview: View the machine's token consumption for a selected time period. At the top of the dashboard, a time range selector allows you to switch between different periods. Below, statistical cards display the Input token, Output token, and Total token usage.
Agent status: The status bar on the right displays the current agent process status and lets you restart hung agent processes.
Session interruption diagnosis: The system automatically identifies issues such as long-running sessions with no output, unresponsive conversations, LLM errors, and agent process crashes. It provides a detailed cause analysis to help you quickly resolve the problem. The session list displays information for each session, such as Session ID, Agent, Model, number of conversations, and token usage. Expanding a session reveals the Conversation ID sub-table, where the Interruption column uses labels like 1 High Risk to flag anomalies. At the bottom, the Interruptions panel shows details of unresolved interruptions, such as an LLM error with the message invalid access token or token expired and status code 401. You can handle these by clicking the Resolve or Hide buttons.
Session details: Click "Details" to view detailed token usage for each session and trace.

In the Interaction trace panel, you can switch between views using the By Trace and By Session tabs at the top. Enter a Trace ID and click Load to view its details. The agent information card on the left shows the name, version, model, and the number of defined tools. The statistics cards on the right display Total steps, Total input tokens, and Total output tokens. Below, the Interaction trace area lists each step of the interaction with role tags, timestamps, and content, including system prompt text and a list of tool definitions such as read, write, edit, and exec.

Model analysis: Analyze the model prompt and thinking process for a user's input to pinpoint the main sources of token consumption.
Token saving: View the current number of saved tokens, click a SESSION ID to view each optimization item, and click Details to view a before-and-after comparison of the content.

On the Token saving page, you can filter queries by time range and agent. At the top, summary cards show Total token consumption (broken down by input/output), Tokens reduced (broken down by tool/MCP), and the Reduction rate (with a rating level). The table below lists the input/output tokens, reduced amount, and reduction rate for each session.

For example, under the MCP output category, you can see the token count before and after optimization for each item (e.g., 734 before, 182 after). When you expand the details, the Original content (on a red background) on the left shows the full JSON, while the Optimized content (on a green background) on the right shows a streamlined result where redundant metadata fields are truncated to save tokens.

Data management

Database management

Automatic capacity limiting and cleanup: The system defaults to a maximum database size of 200 MB to prevent indefinite database growth and excessive disk usage. When the database size reaches this limit, a cleanup process is automatically triggered.

You can customize the maximum size (in MB) using the AGENTSIGHT_GENAI_DB_MAX_SIZE_MB environment variable. For example, to set it to 500 MB:

export AGENTSIGHT_GENAI_DB_MAX_SIZE_MB=500

Clear historical data

To clear historical data, run the following command:

rm -rf /var/log/sysak/.agentsight

Then, restart AgentSight.

Configuration management

Default path: /etc/agentsight/config.json. You can specify a different path with the --config option at runtime.

The complete structure is as follows:

{
 "verbose": 0,
 "log_path": "/var/log/agentsight/agentsight.log",
 "cmdline": {
 "allow": [
 {"rule": ["hermes*"], "agent_name": "Hermes"},
 {"rule": ["python", "hermes"], "agent_name": "Hermes"},
        {"rule": ["node*", "copilot-shell"], "agent_name": "Cosh"},
        {"rule": ["openclaw-gatewa"], "agent_name": "OpenClaw"}
      ],
      "deny": [
        {"rule": ["node*", "webpack"]},
        {"rule": ["node*", "eslint"]}
      ]
    },
    "domain": [
      {"rule": [".openai.com", ".anthropic.com", "*.dashscope.aliyuncs.com"]}
    ]
}

The fields are described in the following table:

Field	Type	Required	Default value	Description
`verbose`	int	No	`0`	`0`: warn-level logs. Any non-`0` value: debug-level logs.
`log_path`	string	No	None (output to stderr)	Log file path.
`cmdline`	object	No	Built-in agent list	Process command-line matching rules.
`cmdline.allow`	array	No	`[ ]`	Process rules allowed for tracing.
`cmdline.deny`	array	No	`[ ]`	Process rules denied for tracing.
`domain`	array	No	`[ ]`	Domain filter rules. Only traffic matching the specified domains is captured.

FAQ

Q1: Why can't I get token consumption data for OpenClaw?
A: AgentSight monitors the openclaw-gateway daemon process. Ensure the client is connected to the gateway. The following error log indicates that device pairing failed:
Gateway agent failed; falling back to embedded: Error: gateway closed (1008): pairing required
We recommend running the command openclaw devices approve to complete the device pairing.

Q2: Why isn't the current session ID displayed on the token saving page, or why are the token savings shown as 0?
A: This can happen for either of the following reasons:

The current version does not support Copilot Shell's AK/SK authentication method.
The session ID is not in the standard UUID format, causing the system to fail to match it.

Q3: Why is the "optimization savings" on the token saving page greater than the difference between the "pre-optimization token count" and the "post-optimization token count"?

A: The agent includes historical messages in the context of each new conversation. Therefore, the current conversation's statistics also include savings from previous messages, causing the cumulative savings to exceed the savings from that single conversation.