# Model Routing

> Intelligent Model Routing dynamically analyzes request content and automatically matches the most suitable model from the candidate set, helping enterprises achieve optimal AI resource allocation in complex scenarios.

## Overview

<span id="model-routing-overview" />

The model-code has a fixed `auto-model` prefix followed by 8 random characters (e.g., `auto-model-abcd1234`). The client calls this model-code → routing dynamically analyzes request content and matches the most suitable model from the candidate set → actual model infers → response (`x-dashscope-resolved-model` marks actual model, retains real token usage).

**Scope**:

| Dimension     | Description                                                                                                           |
| ------------- | --------------------------------------------------------------------------------------------------------------------- |
| Region        | Beijing (`cn-beijing`) + Singapore (`ap-southeast-1`)                                                                 |
| Model type    | Text input only                                                                                                       |
| Call protocol | Only OpenAI-compatible interface (Chat Completions) is supported; DashScope and Anthropic protocols are not supported |
| Billing       | Pay-as-you-go                                                                                                         |
| Throttling    | By actual routed model, at account and workspace level                                                                |

**Key advantages**:

- **Maximize model effectiveness**: Automatically matches the optimal model for each task, ideal for users with extreme quality requirements.
- **Precise cost reduction**: Routes simple tasks to lightweight models and complex tasks to high-end models, avoiding over-provisioning and achieving precise cost-demand alignment.
- **Eliminate model selection difficulty**: No need for manual model evaluation or comparison; the platform auto-selects the optimal solution, drastically lowering the model selection barrier.
- **Minimal learning curve**: Consistent with regular model access; call via model-code with zero code changes.

## Create a model router

<span id="create-model-routing" />

In the [Bailian console](https://modelstudio.console.alibabacloud.com/cn-beijing/model/deploy) under "Model Inference > Dedicated Deployment", click <strong>Deploy New Model</strong> to enter the creation page, and select <strong>Intelligent Model Routing</strong> under "Deployment Mode".

### Prerequisites

- A Bailian workspace is activated.
- The target route model version is published, and the current account has invocation permission for the version and candidate models.

### Steps

1. Enter a <strong>Service Name</strong> (required, up to 50 characters).
2. Under <strong>Deployment Mode</strong>, select "Intelligent Model Routing".
3. In <strong>Route Model Version</strong>, select a published version (maintained by the platform).
4. Under <strong>Routing Strategy</strong>, select "Effect First" or "Cost First".
5. Under <strong>Candidate Model Set</strong>, select candidate models (at least 2).
6. Click <strong>Confirm Deploy</strong> to complete creation.

After deployment, the system generates a globally unique `model-code` (in the form `auto-model-XXXXXXXX`). The `model-code` is not reused after deletion.

### Routing strategy

<span id="routing-strategy" />

The routing strategy determines which model in the candidate set executes each request. Both strategies dynamically match by request content (not fixed routing); the choice is made at creation.

<table>
  <colgroup>
    <col style={{ width: "33.33%" }} />

    <col style={{ width: "33.33%" }} />

    <col style={{ width: "33.34%" }} />
  </colgroup>

  <thead>
    <tr>
      <th><p><strong>Strategy</strong></p></th>
      <th><p><strong>Selection mechanism</strong></p></th>
      <th><p><strong>Suitable scenarios</strong></p></th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td><p>Effect First (EFFECT\_FIRST)</p></td>
      <td><p>Picks the strongest-effect model in the candidate set per request</p></td>
      <td><p>Critical business, complex reasoning, high-quality content generation where quality is stringent</p></td>
    </tr>

    <tr>
      <td><p>Cost First (COST\_FIRST)</p></td>
      <td><p>Intelligently schedules lightweight models while meeting a quality floor</p></td>
      <td><p>Cost-sensitive, high-concurrency, latency- and unit-cost-aware scenarios</p></td>
    </tr>
  </tbody>
</table>

### Candidate model configuration

<span id="model-routing-candidate" />

The candidate model set is the range the routing service can choose from. When creating a model router, select the models to participate in routing from the candidate list in the <strong>Candidate Model Set</strong> area—**at least 2 candidate models** are required, otherwise deployment is blocked.

<Note>RAM users can only select candidate models they have <strong>invocation permission</strong> for—if fewer than 2 are selectable, grant the RAM user model invocation authorization in that workspace first.</Note>

Route model version is the standardized release form of intelligent routing capability; each version specifies the model range and core capabilities supported by the routing system for that stage.
The candidate list is determined by the selected route model version—each version corresponds to one candidate set; cross-version combination is not allowed.
The currently published route model version and candidate models:

<Tabs>
  <Tab title="model-router-2026-0908">
    <table>
      <colgroup>
        <col style={{ width: "20.00%" }} />

        <col style={{ width: "20.00%" }} />

        <col style={{ width: "20.00%" }} />

        <col style={{ width: "20.00%" }} />

        <col style={{ width: "20.00%" }} />
      </colgroup>

      <thead>
        <tr>
          <th><p><strong>Candidate model</strong></p></th>
          <th><p><strong>Context</strong></p></th>
          <th><p><strong>Input</strong></p></th>
          <th><p><strong>Output</strong></p></th>
          <th><p><strong>Cache hit input</strong></p></th>
        </tr>
      </thead>

      <tbody>
        <tr>
          <td><p>`qwen3.8-max`</p></td>
          <td><p>1M</p></td>
          <td><p>12</p></td>
          <td><p>36</p></td>
          <td><p>1.5</p></td>
        </tr>

        <tr>
          <td><p>`qwen3.8-flash`</p></td>
          <td><p>1M</p></td>
          <td><p>0.8</p></td>
          <td><p>2.7</p></td>
          <td><p>0.1</p></td>
        </tr>

        <tr>
          <td><p>`qwen3.7-plus`</p></td>
          <td><p>1M</p></td>
          <td><p>2</p></td>
          <td><p>8</p></td>
          <td><p>0.4</p></td>
        </tr>

        <tr>
          <td><p>`qwen3.7-flash`</p></td>
          <td><p>1M</p></td>
          <td><p>0.2</p></td>
          <td><p>0.8</p></td>
          <td><p>0.04</p></td>
        </tr>

        <tr>
          <td><p>`qwen3.7-max`</p></td>
          <td><p>1M</p></td>
          <td><p>12</p></td>
          <td><p>36</p></td>
          <td><p>2.4</p></td>
        </tr>

        <tr>
          <td><p>`deepseek-v4-pro-0813`</p></td>
          <td><p>1M</p></td>
          <td><p>9</p></td>
          <td><p>27</p></td>
          <td><p>0.9</p></td>
        </tr>

        <tr>
          <td><p>`deepseek-v4-flash-0731`</p></td>
          <td><p>1M</p></td>
          <td><p>3</p></td>
          <td><p>9</p></td>
          <td><p>0.3</p></td>
        </tr>

        <tr>
          <td><p>`Kimi-K3`</p></td>
          <td><p>1M</p></td>
          <td><p>20</p></td>
          <td><p>100</p></td>
          <td><p>2</p></td>
        </tr>

        <tr>
          <td><p>`glm-5.2`</p></td>
          <td><p>1M</p></td>
          <td><p>8</p></td>
          <td><p>28</p></td>
          <td><p>2</p></td>
        </tr>
      </tbody>
    </table>
  </Tab>
</Tabs>

Price unit: CNY per million tokens. The deepseek-v4 series uses busy/idle time-based pricing; the table shows busy-time prices. All candidate models support thinking mode; thinking tokens are billed at output price.

<Note>When selecting candidate models, note each model's quota limits to avoid being throttled when routed to a model due to insufficient quota. Choose models with sufficient quota, or raise the account's RPM/TPM quota for the relevant models before creation.</Note>

## Invoke a model router

<span id="invoke-model-routing" />

Once created, calling a model router is the same as calling a regular model—just replace the `model` parameter in the request with the router's `model-code` (e.g., `auto-model-abcd1234`); the client can integrate with zero code changes.

Example call:

<CodeGroup dropdown>
  ```bash curl
  curl -X POST https://dashscope-intl.aliyuncs.com/compatible-mode/v1/chat/completions \
    -H "Authorization: Bearer $API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "auto-model-abcd1234",
      "messages": [{"role": "user", "content": "Hello"}]
    }'
  ```

  ```python OpenAI SDK
  from openai import OpenAI

  client = OpenAI(
      api_key="YOUR_DASHSCOPE_API_KEY",
      base_url="https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
  )
  resp = client.chat.completions.create(
      model="auto-model-abcd1234",
      messages=[{"role": "user", "content": "Hello"}],
  )
  print(resp.choices[0].message.content)
  print(resp.model)  # the actual executing model name
  ```

  ```python DashScope SDK
  # DashScope SDK (install first: pip install dashscope)

  from dashscope import Generation

  resp = Generation.call(
      model="auto-model-abcd1234",
      api_key="YOUR_DASHSCOPE_API_KEY",
      messages=[{"role": "user", "content": "Hello"}],
  )
  print(resp.output.choices[0].message.content)
  ```

  ```javascript Node.js
  // OpenAI SDK (install first: npm install openai)

  const OpenAI = require("openai");

  const client = new OpenAI({
    apiKey: process.env.DASHSCOPE_API_KEY,
    baseURL: "https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
  });
  const resp = await client.chat.completions.create({
    model: "auto-model-abcd1234",
    messages: [{ role: "user", content: "Hello" }],
  });
  console.log(resp.choices[0].message.content);
  console.log(resp.model); // the actual executing model name
  ```
</CodeGroup>

**Response**: the `x-dashscope-resolved-model` response header holds the actually-called model's identifier (use it to verify routing took effect); the body retains the actual executing model name and real token usage, and billing is by the actually-routed model.

**Billing**: Intelligent routing bills by the actual model each request is routed to; the routing model itself is free.

### Auto-failover

<span id="model-routing-failover" />

Model Routing has a built-in auto-failover, enabled by default with no configuration. When the target model is unavailable, the routing service reissues the whole request with another candidate model, not switching mid-inference.

The following do NOT trigger auto-failover; errors pass through with the actual model's original error info:

- 429 (rate limit)
- 400/403/404/500 and other deterministic errors (switching models would likely fail too)
- Timeout or network errors
- Errors after streaming has started returning content

<Note>auto-failover does not double-bill: model A → auto-failover → model B succeeds, only model B is charged.</Note>

### Real-time filtering rules

The routing service applies real-time filtering to the candidate set:

- Filters out models the current account has no permission to call.
- Filters out models that are offline.
- Filters out models incompatible with the current request's parameters (e.g., if the request carries `response_format`, only candidate models supporting that parameter are routed to).

The actually-selectable subset for a single request changes dynamically with permissions, model state, and request parameters.

## Usage limits

<span id="usage-limits" />

| Limit         | Description                      |
| ------------- | -------------------------------- |
| Call protocol | OpenAI-compatible interface only |

Protocol support:

| Protocol                                       | Supported                                                                                                                                                   |
| ---------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |
| OpenAI-compatible interface (Chat Completions) | ✓                                                                                                                                                           |
| DashScope protocol                             | ✗                                                                                                                                                           |
| Anthropic protocol                             | ✗                                                                                                                                                           |
| Request type                                   | Text Chat Completions input only; image, video input, image generation, Batch inference, Embedding/Rerank and other non-generation interfaces not supported |
| Context length                                 | Upper bound = the minimum context window of the candidate-set models; over-long context errors directly without triggering auto-failover                    |
| Throttling                                     | RPM/TPM throttled by actual routed model at account and workspace level; on failover, only the final successful model's consumption is counted              |

### Parameter notes

- `response_format`: Routes only to candidate models supporting this parameter.
- `enable_search`: Routes only to candidate models supporting this parameter .
- `reasoning_effort`: Routes only to candidate models supporting this parameter by default; supported enum values: `none`, `minimal`, `low`, `medium`, `high`, `xhigh`, `max`; non-minimum tiers route only to non-thinking models.
- `enable_thinking`: Routes only to candidate models supporting thinking mode .

In intelligent routing mode, the following parameters are not supported; setting them causes a runtime error:

| Parameter             | Description                |
| --------------------- | -------------------------- |
| `top_logprobs`        | Limit on returned logprobs |
| `logit_bias`          | Token bias                 |
| `stop`                | Stop sequence              |
| `tool_choice`         | Tool choice                |
| `parallel_tool_calls` | Parallel tool calls        |
| `logprobs`            | Whether to return logprobs |
| `top_p`               | Nucleus sampling           |
| `temperature`         | Sampling temperature       |
| `presence_penalty`    | Presence penalty           |
| `n`                   | Number of generations      |
| `thinking_budget`     | Thinking budget length     |

### Cache

Intelligent routing does not support explicit cache; it only supports implicit cache provided by the actual model called.
Cache may only be hit when a request is routed to the same model and the prompt prefix meets that model's cache conditions.
Since different requests may be routed to different models, intelligent routing does not guarantee cache hit rate.
For details on how caching works and supported models, see [Context Cache](/help/en/model-studio/context-cache).

## Manage route service

<span id="manage-model-router" />

### View detail

After deployment, click the service name in the Dedicated Deployment list to enter the detail page:

- **Basic info**: service name, `model-code`, running status, routing strategy tag.
- **Route configuration**: route model version, routing strategy, candidate set and candidate model list.
- **Billing**: pay-as-you-go, billed by the actual model's unit price.
- **Deploy config**: each candidate model's TPM/RPM throttle quota.

### Modify route config

Click "Edit" in the Route Configuration area to modify route version, strategy, and candidate models. Switching version changes the candidate set, taking effect within a few minutes without affecting live requests.

After selecting a specific version, online routing behavior remains long-term stable; please conduct a thorough evaluation before upgrading to avoid business impact. Current version: `model-router-2026-0908`. Route versions are not auto-updated.

### Delete service

Delete the model routing service from the Dedicated Deployment list. `model-code` is not reused after deletion.

## Routing effect

<span id="monitoring" />

This chapter covers routing effect metrics and monitoring. Effect metrics are viewed on the detail page's "Routing effect" tab; call monitoring is viewed on the Model Monitoring page.

### Routing metrics

View the following metrics on the detail page's "Routing effect" tab. The displayed metrics are estimates, for reference only; not for billing settlement; actual billing is based on "Cost and Usage":

<table>
  <colgroup>
    <col style={{ width: "25%" }} />

    <col style={{ width: "45%" }} />

    <col style={{ width: "30%" }} />
  </colgroup>

  <thead>
    <tr>
      <th><p><strong>Metric</strong></p></th>
      <th><p><strong>Meaning</strong></p></th>
      <th><p><strong>Anomaly action</strong></p></th>
    </tr>
  </thead>

  <tbody>
    <tr><td><p>Total routing requests</p></td><td><p>Total requests received (including failover-succeeded)</p></td><td><p>Investigate upstream throttling/anomalies on spikes</p></td></tr>
    <tr><td><p>Routing success rate</p></td><td><p>Forwarded requests ÷ total requests × 100%</p></td><td><p>On drop, check candidate model availability</p></td></tr>
    <tr><td><p>Actual model cost (estimated)</p></td><td><p>Full cost estimated at actual model list price (tiered billing: lowest tier; peak-valley: peak price; excludes free quota; deviates from actual billing)</p></td><td><p>On rise, check routing strategy and candidate set</p></td></tr>
    <tr><td><p>Baseline model cost (estimated)</p></td><td><p>Hypothetical cost if all handled by the baseline model (same estimation methodology)</p></td><td><p>—</p></td></tr>
    <tr><td><p>Savings</p></td><td><p>Baseline cost − actual cost (positive means routing saved cost)</p></td><td><p>—</p></td></tr>
    <tr><td><p>Billed request count</p></td><td><p>Successful requests that incur charges</p></td><td><p>—</p></td></tr>
    <tr><td><p>Spend share</p></td><td><p>Single-model cost ÷ total cost</p></td><td><p>On anomaly, check candidate-set distribution</p></td></tr>
  </tbody>
</table>

### Cost notes

Cost analysis data has approximately a 1-hour delay and is not real-time.
The above costs are all estimated by catalog price as full cost, excluding account discounts and free quotas, for reference only to evaluate routing savings.
Tiered billing models use the lowest-tier catalog price; peak-valley pricing models use peak prices.
Actual billing can be viewed in "Cost and Usage".

### Model monitoring

For more granular call monitoring, go to the Model Monitoring page.
It shows model monitoring and logs at the `auto-model` dimension, not individual monitoring of the actually-routed models.