AI Gateway Serverless uses Agent resources to centrally manage the services that Agents expose and the models that they call. Agent resources suit teams that run multiple Agents, such as Dify, Claude Code, DeepSeek Harness, and Codex. This topic describes how to create, configure, and connect an Agent, and how to view its access paths, usage, and cost.
Agent management is supported only on the Standard and Enterprise editions of the new Serverless gateway. Legacy Serverless gateways and instance-based gateways are not supported. For differences among gateway types, see Gateway editions.
Agent resources and supported capabilities
An Agent is an AI Gateway management resource for centrally maintaining access configurations and aggregating observability data. Depending on the Agent type, you can configure one or both of the following access capabilities:
Agent Access — Expose an Agent service through an Agent API, and centrally configure routes, consumer authentication, policies, and plugins. Use this capability when users or other applications call the Agent.
Model Access — Bind the Model APIs used by an Agent to a consumer that represents the Agent identity, and centrally configure the model protocol, endpoint, authorization, policies, and plugins. Use this capability when an Agent calls models.
The following table lists the access capabilities supported by each Agent type.
Agent type | Agent Access | Model Access | Scenarios |
Model Studio | Supported | Not supported | Expose Model Studio applications through AI Gateway. |
Dify | Supported | Supported | Manage inbound access and model calls for Dify applications. |
Claude Code | Not supported | Supported | Manage model calls for Claude Code. |
DeepSeek Harness | Not supported | Supported | Manage model calls for DeepSeek Harness. |
Codex | Not supported | Supported | Manage model calls for Codex. |
Custom | Supported | Supported | Configure Agent Access, Model Access, or both as needed. |
After you configure access capabilities and generate call data, you can use the following observability capabilities.
Capability | Purpose | Questions it helps answer |
Session Usage Statistics | View tokens, credits, duration, cache hit ratio, and tool calling at the session, turn, and model call levels. | Which sessions have high usage or long duration? Which turn or model call accounts for most token or credit consumption? |
Agent statistics | View request volume, token consumption, error rate, model fallback rate, latency, and model and consumer distribution by Agent. | Which model or consumer has high usage? Are error rates and latency abnormal? Does model fallback occur frequently? |
Prerequisites
Prepare the following items based on the capabilities that you plan to use:
Gateway instance — A Standard or Enterprise edition AI Gateway instance on the new Serverless gateway. For instructions, see Create a gateway instance.
Agent Access — An accessible Agent backend service.
Model Access — A consumer that has access permissions for the Model API that you want to bind.
Session Usage Statistics — Session analysis enabled for the gateway. Session analysis requires gateway access logs.
Agent statistics (Model Access) — Daily analysis subscribed for the Agent. The system generates analysis results daily.
Create and manage Agents
Create an Agent
The Agent name and Agent type cannot be changed after the Agent is created. You can configure access capabilities during creation or afterward.
Log on to the AI Gateway console. Select the target region and a new Serverless AI Gateway instance.
In the left-side navigation pane, click Agent Management.
Click Create Agent.
Configure the following basic settings.
Parameter
Description
Agent Name
The unique name of the Agent in the current gateway. The name can contain letters, digits, hyphens (-), and underscores (_), and must not exceed 64 characters.
Agent Type
Select Model Studio, Dify, Claude Code, DeepSeek Harness, Codex, or Custom. The Agent type determines which access capabilities you can configure later.
As needed, expand Agent Access Configuration (Optional) or Model Access Configuration (Optional) and complete the relevant settings. You can also click OK and configure these settings after you create the Agent.
After the Agent is created, the console opens the Agent Topology tab, where you can configure the access capabilities of the Agent.
Configure Agent Access
Agent Access is suitable when you expose an Agent service to users or applications. Configure the access endpoint, routes, consumer authentication, policies, and plugins through the Agent API to centrally manage callers and the exposed service. Before you start, ensure that the Agent backend service is accessible. Model Studio, Dify, and Custom types support this capability.
Open the configuration entry. When you create an Agent, expand Agent Access Configuration (Optional). After you create an Agent, configure the Agent Access area on the Agent Topology tab.
Select an existing domain name, or click Add Domain Name to create one. Set the access path to route the Agent API to the corresponding Agent backend service.
To verify caller identities, enable authentication in the Consumer Authentication card and grant permissions to consumers or consumer groups.
To govern requests, add the required policies or plugins in the Policies and Plugins card.
Confirm that the Agent API, consumer authentication, policies, and plugins are configured correctly.
After configuration is complete, callers can access the Agent by using the domain name and access path of the Agent API.
Agent Topology shows the relationships among Agent APIs, Model APIs, consumers, policies, and plugins, which helps you understand the access paths of an Agent. Manage Agent APIs, services, routes, domain names, and consumers on their respective resource pages. The topology updates when these resources change.
Configure Model Access
Model Access is suitable when an Agent calls models through AI Gateway. After you bind a Model API to a consumer that represents the Agent identity, you can centrally manage the endpoint, protocol, authorization, policies, and plugins, and attribute model usage to the Agent. When multiple Agents share a model service, you can still view their calls separately. A Model API can be associated with multiple Agents, but within the same Model API, each consumer can be bound to only one Agent. Dify, Claude Code, DeepSeek Harness, Codex, and Custom types support this capability.
Open the configuration entry. When you create an Agent, expand Model Access Configuration (Optional). After you create an Agent, configure the Model Access area on the Agent Topology tab.
Select an existing Model API, and confirm the model protocol, BasePath, and domain name. If no Model API is available, create one from the page entry. The Agent client must use the endpoint and BasePath displayed here to access models. For information about managing Model APIs, see Manage Model APIs.
In the Consumer Binding card, bind one or more consumers. AI Gateway attributes model calls to the corresponding Agent based on this binding. Binding a consumer does not automatically grant permissions, so ensure that the consumer has access permissions for the selected Model API. If no consumer is available, click Create Consumer. For instructions on creating a consumer, see Create a consumer.
To record model requests or enable content security protection, enable or add the relevant policies in the Policies and Plugins card, such as AI Request Log and AI Security Protection.
Confirm that the Model API, consumer binding, policies, and plugins are configured correctly.
After configuration is complete, connect the Agent client and initiate a model call to verify that AI Gateway records the call and attributes it to the Agent.
Edit an Agent
You can change the access configurations and associated resources of an existing Agent, but not its name or type.
On the Agent Management page, find the target Agent and click Edit in the Actions column. You can also click the Agent name to open the topology and adjust the related settings.
Modify the Agent Access or Model Access configuration.
Click OK to save the changes.
Delete an Agent
On the Agent Management page, find the target Agent and click Delete in the Actions column.
Read the deletion prompt, and then confirm the deletion.
Deleting an Agent removes its relationships with related resources and revokes related permissions. It does not delete the associated Agent APIs, Model APIs, services, domain names, consumers, or consumer groups. If you no longer need these resources, manage them separately on their respective resource pages.
Connect an Agent
This section applies to Agents that call models through AI Gateway. After you configure Model Access, update the running Agent client so that its model requests pass through AI Gateway. After the connection is complete, you can use consumer authentication, policies, and plugins, and view Agent statistics and Session Usage Statistics. If your Agent type supports only Agent Access, callers access the Agent through the domain name and access path of the Agent API after you configure Agent Access.
Get connection information
On the Agent Management page, click the target Agent name.
On the Agent Topology tab, click Agent Connection on the Agent card.
In the connection guide, select the API protocol and endpoint, and enter the model to access, model display name, and consumer for authentication.
Use the guide generated on the page to obtain the following information:
The API address, which consists of the endpoint and Model API BasePath
The protocol used by the Model API
The API key of the selected consumer
The model name and display name that the Agent client must use
Configure and verify the client
The console generates connection steps based on the Agent type. Follow the connection guide generated on the page for the specific configuration. DeepSeek Harness, for example, supports the following two configuration methods, and the connection guide describes both:
Manual platform configuration — In DeepSeek Harness, choose Settings > Models > Add Custom Provider. Enter the Provider ID, display name, API address, API protocol, and API key as instructed in the connection guide, and then add the model.
Local configuration file — Merge
~/.dsh/settings.yamlas instructed in the connection guide, and set the required environment variables.For other Agent types, follow the connection guide generated in the console. If the client does not provide a dedicated guide, use its custom endpoint or custom model provider configuration, and enter the API address, protocol, consumer API key, and model name that AI Gateway generates.
After configuration is complete, initiate a model call from the Agent client. If the model returns a response and you can find the call in AI Gateway, the connection is successful. If no data appears on an observability page, verify that session analysis or daily analysis is enabled as described in Prerequisites.
A consumer API key is an access credential. Do not store it in a code repository or public document. Use the client credential management feature or environment variables instead.
Agent observability
Session Usage Statistics
Session Usage Statistics is located under FinOps and shows only Agents for which Model Access is configured. Use it to view current-day or historical session usage and analyze a conversation. The session dashboard shows the session count, tokens, credits, and duration by Agent and consumer. Session details let you drill down into turns and model calls to view tool calling, call results, time to first token (TTFT), and other data. The data hierarchy is as follows:
Session — A complete conversation between a user and an Agent. A session can contain one or more turns.
Turn — A user question and the process in which the Agent handles it. One turn can generate multiple model calls because of tool calling, retries, and other reasons.
Model call — A model request initiated within a turn. You can view the model name, input and output types, finish reason, TTFT, tokens, and duration for each call.
Enable session analysis
In the left-side navigation pane of the target AI Gateway instance, choose FinOps > Session Usage Statistics.
If the page indicates that the feature is not enabled, follow the on-screen instructions to enable session analysis. Session analysis requires gateway access logs. If standard access log delivery is already enabled, the system uses the existing configuration. Otherwise, the page guides you through the configuration.
After you enable session analysis, only newly generated calls are analyzed. Historical sessions generated before you enable it are not added.
AI Gateway does not charge separately for logs. Simple Log Service (SLS) charges based on actual usage. For billing information, see Enable log shipping for an AI Gateway.
View the session dashboard
On the Session Usage Statistics page, select an Agent and a date.
Click the Session Dashboard tab.
View the following information.
Analysis area | Description |
Overview | The number of sessions and consumers for the selected Agent on the current day. |
Session performance and resource consumption distribution | The P50 and P90 values of session tokens, session duration, and session credits. |
Session rankings | Sessions with high usage, ranked by tokens, total duration, and credits. |
Consumer usage rankings | Consumer usage, ranked by session count, tokens, and credits. |
The current-day session dashboard has a data latency of about 5 minutes. Because the session dashboard and session list can be updated at different times, their data can be inconsistent for a short period.
View the session list and details
Click the Session List tab.
Enter a session name or session ID to search for the target session.
View the consumer, tokens, credits, cache hit ratio, tool call count, total model call duration, total session duration, and start time of the session.
In the Actions column for the target session, click View Details.
On the details page, view the following information:
Basic session information, and the token, credit, and duration distribution across turns
Turn rankings by tokens, duration, and credits
The turn timeline and model call details within each turn
The model name, input and output types, finish reason, tool calling, call result, TTFT, tokens, credits, start time, and duration
Total session duration is the actual elapsed time from the first request in a session to the end of the last request, including waiting time between turns. Total model call duration is the sum of the durations of all model calls in the session. To troubleshoot a slow session, use the turn timeline and model call details to determine whether time is mainly spent on model calls, tool calling, or waiting between turns.
Agent statistics
On the Agent Management page, find the target Agent and click Statistics in the Actions column. You can also open the Agent details page and click the Statistics tab. The statistics page contains the Model Access and Agent Access tabs. The page displays statistics only after you configure the corresponding access capability and generate call data.
View Model Access statistics
Model Access statistics help you understand the overall daily model call activity of an Agent. You can view request count, tokens, error rate, fallback rate, and latency, and break down usage by model and consumer. Intelligent insights identify anomalies and provide troubleshooting recommendations to help with usage attribution and stability analysis.
On the Statistics tab of the Agent, click Model Access.
Turn on Subscribe to Daily Analysis. The analysis data comes from
apig-access-login Simple Log Service (SLS), and the system generates analysis results daily.Select the analysis date that you want to view.
View the following information.
Analysis area | Description |
Core metrics | Total requests, total tokens, error rate, fallback rate, and P95 request latency. |
Intelligent insights | Anomalies in token consumption, error rate, fallback rate, or latency for the selected date, together with troubleshooting recommendations. When there are no requests on the current day, latency and error rate display as 0 and cannot be used to assess performance. |
Traffic structure | The proportion of model requests, token consumption by model, consumer distribution, and request trends over time. |
Cost attribution | The proportion of input and output tokens, and the distribution of text, image, reasoning, cache, and other token types. |
Stability and latency | Time to first token, generation time per output token, overall request latency, HTTP 5xx, fallback, and response flags. |
Consumer details | Requests, request proportion, tokens, token proportion, and error rate for each consumer. |
Model Access statistics are separated by Agent. Even when multiple Agents use the same Model API, you can view the model call data of each Agent separately based on the consumer-to-Agent binding.
View Agent Access statistics
Agent Access statistics apply when services are exposed through an Agent API. View caller request volume, error rate, and latency by Agent to understand external service usage and stability.
On the Statistics tab of the Agent, click Agent Access.
Select an analysis date to view Agent API access statistics.
The page does not show related data if the current Agent type does not support Agent Access or if Agent Access configuration is incomplete.
FAQ
Why is no statistical data generated immediately after I create an Agent?
Creating an Agent saves only its name and type. You must also configure access capabilities and route actual requests through AI Gateway before the statistics page generates data. Before you view Model Access statistics, subscribe to daily analysis. Before you view Session Usage Statistics, enable session analysis.
Why can't I find an Agent in Session Usage Statistics?
Session Usage Statistics shows only Agents for which Model Access is configured. First associate a Model API and bind a consumer in Agent Topology, and then initiate model calls through that consumer.
Does a latency of 0 in Model Access statistics mean that performance is good?
Not necessarily. If there are no requests on the selected date, latency, error rate, and other metrics display as 0, but they cannot be used to assess performance. First confirm the total request count and request trend, and then review latency and stability.
Why is total session duration much greater than total model call duration?
Total session duration includes waiting time between turns, while total model call duration counts only the duration of individual model calls. Use the turn timeline in the session details to determine whether time is mainly spent on model calls, tool calling, or waiting between turns.