Edge Security Acceleration:AI Gateway

Features

AI Gateway acts as a proxy layer between your applications and AI service providers, providing the following core features:

• Unified access: Connect to multiple AI service providers through a singleendpoint. Compatible with the OpenAI API standard, it allows you to switch between different models without modifying your existing code.

• Performance optimization: Its intelligent cache returns results for repeated requests, reducing response latency and lowering upstream API call costs.

• Security: Provides multiple layers of security, including rate limiting, IP allowlist, and API key authentication, to protect your AI services from abuse and attacks.

• High availability: Ensures business continuity with automatic failover and request retries. When the primary provider's service is unavailable, AI Gateway automatically switches to a backup provider.

• Observability: Provides real-time monitoring of request count, token usage, response time, and cost statistics. It also offers detailed request logs and audit trails.

How it works

AI Gateway is deployed on the global edge nodes of ESA. The request processing flow is as follows:

1. Request reception: The client application sends an API request to the nearest edge node through a unifiedendpoint.

2. Authentication: The edge node authenticates requests using methods such as an API token or IP allowlist.

3. Request routing: AI Gateway forwards the request to a specified AI service provider based on your configured routing rules, such as Alibaba Cloud Model Studio or OpenAI.

4. Result caching: The edge node caches the upstream response and returns it to the client.

5. Monitoring and logging: RecordsSampled Logs and metrics such asRequests,Token Usage, andError Count.

Routing mechanism: Use themodelparameter to specify the target AI model in the format{provider}/{model-name}. For example:

• openai/gpt-4o: Calls the OpenAI GPT-4o model.

• deepseek/deepseek-chat: Calls the DeepSeek chat model.

• qwen/qwen-plus: Calls the Qwen Plus model from Alibaba Cloud Model Studio.

Use cases

import OpenAI from "openai";

const client = new OpenAI({
  defaultHeaders: {
    "esa_aigw_auth": `Bearer ESA_API_TOKEN`,
  },
  baseURL: "https://YOUR_DOMAIN/v1",
});

const response = await client.chat.completions.create({
  model: "qwen/qwen-plus",  // Specify the model to use
  messages: [{ role: "user", content: "What is Edge Security Acceleration?" }],
});

import OpenAI from "openai";

const client = new OpenAI({
  defaultHeaders: {
    "esa_aigw_auth": `Bearer ESA_API_TOKEN`,
    "x-ai-route-metadata": "YOUR_IFELSE_CONDITION_RULE",  // Required when an If...Else condition is configured
  },
  baseURL: "https://api.example.com/v1",
});

const response = await client.chat.completions.create({
  model: "dynamic/ai-test",  // Route configuration
  messages: [
    { role: "user", content: "Hello!" }
  ],
});

console.log(response.choices[0].message.content);

Usage notes

• API key management: You must configure your upstream providers' API keys in AI Gateway. Securely store and rotate your keys regularly. The keys are stored encrypted inESA.

• Model compatibility: The unified interface follows the OpenAI API standard. Some provider-specific features may not be supported.

• Monitoring data latency: Monitoring metrics may have a delay of 1 to 5 minutes. For scenarios that require high real-time visibility, we recommend using log analysis as a supplement.

Procedure

Route configuration

You can configure routing rules to automatically match requests to models based on their characteristics. You can also add failover rules to automatically switch to a backup provider or another model from the same provider if the primary model service becomes unavailable.

1. In the AI Gateway application, select theRouting Configurationtab, clickAdd Route, and enter a customNameon the page that appears.

2. After creating the route configuration, find it in the route list and clickActionsin theEditcolumn.

3. On the edit page, you can configure the model route using the visual editor:

   1. Configure routing conditions: Click the add iconbetween theStartandModel Servicenodes in the visual workflow. ClickAdd Conditionto automatically generate anIf...Elseflow. Click theIf...Elsebutton in theConfigureflow, and then clickAdd Conditionin the panel that appears on the right.

      • metadata.: Enter the key to be evaluated, such astest-key.

      • Logical operator: Select a logical operator, such as equals (==) or not equals (!=).

      • Value: Enter the value for metadata., such as ok.

      Note

      By default, the flow uses the primary provider if the condition evaluates totrueand the backup provider if it evaluates tofalse. You can manually edit the model configuration to change this behavior.

   2. Configure the model service: Click the add iconon both thetrueandfalsebranches. ClickAdd Model Service(or clickAdd Conditionto add a nested condition). Then, clickModel Serviceon theConfigurenode and add the model information:

      • AI Service Provider: Select an AI service provider from the dropdown list, such asQwen.

      • Model Name: Select the corresponding model from the dropdown list, such asqwen3-max.

      • API key: Select anadded API key, such asqwen-api-key.

      You can clickAdd Fallback Modelto configure a backup model for failover.

4. After completing the configuration, clickSave and Deploy. Subsequent requests will be routed based on matching the routing conditions against thex-ai-route-metadataheader.

Analytics and logs

ESAcollects key data for all requests that pass through AI Gateway to help you analyze the operational status of your AI applications.

1. In the AI Gateway application, select theAnalyticstab.

2. Click the filter to select a time range. The dashboard displays the following information:

   • Requests: The number of requests sent through this AI Gateway during the selected time period.

   • Token Usage: The estimated number of tokens used by this AI Gateway during the selected time period.

   • Error Count: The number of error requests from this AI Gateway during the selected time period.

   • Sampled Logs: Detailed request and response information, includingEdge Status Code,Time,Model,Request, andResponse.

Supported providers

AI Gateway supports the following mainstream AI service providers. Support for other providers will be gradually introduced.

Billing

AI Gateway is available on aFree Versionby default. This tier is suitable for most developers and personal projects. A paid tier for enterprise users with production applications will be introduced in the future.