Global Accelerator:Use Global Accelerator to proxy and schedule AI models

Configure AI service endpoint groups

Log on to the Global Accelerator console and click Create Standard Pay-as-you-go Instance.

1. Basic Instance Configuration: Select an Accelerated IP Address Type as needed.

2. Configure Acceleration Area: Set the Acceleration Area and Assign Bandwidth.

3. Configure Listeners: Set the Intelligent Routing. For Protocol, select HTTPS. Set Port to 443. For SSL Certificate Configuration, choose the uploaded server certificate that matches your custom domain.

4. Configure an Endpoint Group:

   After creating the GA instance, go to the listener details page, select the Endpoint Group tab, and click Add Endpoint Group. Follow these steps to configure AI service endpoint groups in different regions. Each endpoint group supports only one backend service of type AI service. You cannot add other backend services after adding this type.

   • Region: Choose the region closest to your AI provider’s available service.

     If your business involves acceleration between the Chinese mainland and China (Hong Kong), Macao (China), Taiwan (China), or other countries and regions, read and sign the Compliance Commitments Regarding Cross-border Data Transfers as prompted.

   • Backend Service Type: Select AI Service.

   • LLM Provider: Currently supports Alibaba Cloud Model Studio and OpenAI Compatible.

   • Service Endpoint: Automatically filled after selecting a provider. Alibaba Cloud Model Studio defaults to https://dashscope.aliyuncs.com/compatible-mode/v1. OpenAI Compatible defaults to https://api.openai.com/v1. Modify as needed if using a self-hosted OpenAI-compatible service.

   • API key: Enter your provider’s API key. You can enter multiple keys (separated by line breaks). GA rotates among them to improve availability.

     This API key authenticates requests with your AI provider. Keep it secure and avoid exposure.

   • Weight (Valid values: 0 to 255): Each endpoint group supports only one AI service backend type. The weight is fixed at 255 and cannot be changed.

5. Configuration Review: Submit your settings.

Configure forwarding rules

1. After creating the GA instance, click the instance ID, select the Listeners tab, click the listener ID, then click the Forwarding Rule tab, and click Add Forwarding Rule.

2. Configure forwarding rules to route requests for different models to their respective endpoint groups:

   • Name: Enter a custom name.

   • If (Matching All Conditions): Select AI Model and configure as follows:

     • Protocol: Fixed as OpenAI Compatible.

     • Model Name (Optional): Use Glob syntax to match model names. For example, qwen-* matches all Qwen series models, and * matches all models. You can add multiple model names; they are evaluated with an "OR" relationship.

   • Then: Select Forward and choose the corresponding AI service endpoint group.

3. Click OK to create the forwarding rule.

4. To route different models to multiple providers, repeat these steps to create separate forwarding rules for different Glob model names. For example, client requests with model=qwen-plus and model=gpt-4o hit different rules and route to their respective providers.

If no model name is specified, the condition matches all OpenAI-compatible requests. If specified, it matches only the named models. Drag rules to adjust priority.

Configure DNS resolution and verify

1. On the instance details page, copy the GA accelerated domain name (CNAME). In your custom domain’s DNS provider, add a CNAME record pointing your custom domain to the GA accelerated domain.

   DNS propagation typically takes minutes, depending on your DNS provider’s TTL setting.

2. Use your custom domain to call the AI service API and verify the integration.

   Taking the model name <kbd data-init-id="s2-cond-kbd-1" data-tag="userinput" id="21f447ae1e2hv">qwen-*</kbd> as an example.

   curl -X POST https://your-domain.com/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{
       "model": "qwen-plus",
       "messages": [{"role": "user", "content": "Hello"}]
     }'

   On success, the model field in the response is qwen-plus, confirming the request was correctly routed to Alibaba Cloud Model Studio.

Configure multi-region endpoint groups

After creating a GA instance, click the instance ID, select the Listeners tab, click the listener ID, select the Endpoint Group tab, and click Add Endpoint Group to configure AI service endpoint groups in different regions.

Configure forwarding rules

After configuration, verify: when the primary endpoint group fails, requests automatically fall back in sequence to other regional endpoint groups, which respond via their respective providers or regions.

1. On the listener details page, click the Forwarding Rule tab, then click Add Forwarding Rule or Edit an existing rule.

2. Configure the forwarding rule:

   • If (Matching All Conditions): Select AI Model and configure as follows:

     • Protocol: Fixed as OpenAI Compatible.

     • Model Name (Optional): Use Glob syntax. For example, qwen-* matches all Qwen series models, and * matches all models. Add multiple names with an "OR" relationship.

   • Then: Configure two actions in order:

     1. Forward: Select an endpoint group in one region.

     2. Click Add Action, select Rollback, and choose an endpoint group in another region.

     Fallback action must come after the Forward To action. Each forwarding rule must contain exactly one action of type Forward To, Redirect To, or Return Fixed Response.

3. Click OK to create the rule. Repeat for different Glob model names as needed.

If no model name is specified, the condition matches all OpenAI-compatible requests. If specified, it matches only the named models. Drag rules to adjust priority.