Model API supports custom routes, which allow you to perform fine-grained routing based on request features. This topic describes how to manage custom routes.
Create a route
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
From the Routes, click Create routes. In the Create routes panel, configure the parameters:
Route Name: The custom name of the route.
Path: The custom route path of Model API. The following paths are supported:
/v1/chat/completions,/v1/completions,/v1/embeddings, and/v1/fine-tuning.More Matching Rules: If you want to create multiple routes with the same path, you must configure Method, Header, and Query Parameters to distinguish between the routes.
Service Type: Single Service and Multiple services (by proportion) are supported.
Single Service: Only one model is deployed and runs in a service instance. This type is suitable for scenarios that require focus on a single task or model.
Multiple services (by proportion): Multiple models are deployed and run in the same service instance. Resources and requests are allocated based on the specified ratios. This type is suitable for scenarios that require simultaneous processing of multiple tasks or models.
Services: Select a service from the service list.
Fallback: After you enable this feature, you can add a fallback service. The services are executed in descending order.
NoteYou can use the same service to create multiple fallback policies.
Confirm the configuration and click Add.
Edit a route
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click Routes and select the destination custom route.
Click Edit to modify the parameters. For more information about the parameters, see Create a route.
Confirm the configurations and click OK.
Publish a route
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click Routes and select the destination custom route.
Click Publish Route. In the confirmation message, click OK.
Unpublish a route
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click Routes and select the destination custom route.
Choose . In the confirmation dialog box, click Unpublish.
Delete a route
You can delete only unpublished or offline routes. To delete a published route, you must first unpublish the route.
Log on to the AI Gateway console and choose Instance. In the top menu bar, select a region, then click the target instance ID.
In the navigation pane on the left, choose Model API, then click the target API name to go to the API Details page.
Click Routes and select the destination custom route.
Choose . In the confirmation dialog box, click Delete.