AI Gateway provides different QPS and client connection limits for each instance specification, and sets quota limits on each type of resource. This topic lists the capacity specifications of dedicated instances, together with the resource quotas of both Serverless and dedicated instances, for reference when you purchase or upgrade an instance.
Capacity specifications
The following table lists the QPS and maximum client connections for each dedicated instance specification:
|
Instance type |
QPS |
Client connections |
|
aigw.small.x1 |
1500 |
20000 |
|
aigw.small.x2 |
3000 |
40000 |
|
aigw.small.x4 |
6000 |
80000 |
|
aigw.medium.x1 |
12000 |
160000 |
|
aigw.medium.x2 |
24000 |
320000 |
|
aigw.medium.x3 |
36000 |
480000 |
|
aigw.large.x1 |
48000 |
640000 |
|
aigw.large.x2 |
96000 |
1280000 |
|
aigw.large.x3 |
144000 |
1920000 |
|
aigw.large.x4 |
192000 |
2560000 |
Quotas
AI Gateway sets an upper quota limit on each type of resource. Quotas fall into two categories: global quotas, which apply at the region level, and instance quotas, which vary by instance edition and specification tier.
Global quotas
The following table lists quotas that apply across all AI Gateway instances in a region.
|
Quota Item |
Quota |
|
Instances per Region |
100 |
|
Consumer Groups per Region |
1000 |
|
Consumers per Group |
500 |
|
Consumers per Region |
100000 |
Instance quotas
AI Gateway is available in two editions: Serverless and dedicated. Dedicated instances are grouped into three tiers by specification name prefix: Small (aigw.small.*), Medium (aigw.medium.*), and Large (aigw.large.*). The following table lists the instance quotas for each edition and tier. The Number of consumers and Consumer authorization rules are listed as separate quota items and are counted and validated separately:
|
Serverless |
Small |
Medium |
Large |
|||||||||||
|
Specifications |
Serverless (Legacy) |
Serverless (Standard) |
Serverless (Enterprise) |
aigw.small.x1 |
aigw.small.x2 |
aigw.small.x4 |
aigw.medium.x1 |
aigw.medium.x2 |
aigw.medium.x3 |
aigw.large.x1 |
aigw.large.x2 |
aigw.large.x3 |
aigw.large.x4 |
|
|
MCP servers |
100 |
20 |
100 |
500 |
600 |
800 |
1000 |
1200 |
1600 |
2000 |
2200 |
2400 |
2600 |
|
|
Total online routes (including Model API routes and Agent API routes) |
2500 |
200 |
2500 |
5000 |
6000 |
7000 |
7500 |
8000 |
9000 |
10000 |
12000 |
14000 |
16000 |
|
|
Published domains |
100 |
20 |
100 |
200 |
300 |
400 |
500 |
600 |
800 |
1000 |
1200 |
1400 |
1600 |
|
|
Services |
200 |
50 |
200 |
800 |
1200 |
1600 |
2000 |
2500 |
3000 |
4000 |
4500 |
5000 |
6000 |
|
|
Number of consumers |
500 |
100 |
500 |
2000 |
4000 |
6000 |
8000 |
12000 |
16000 |
20000 |
27500 |
35000 |
42500 |
|
|
Consumer authorization rules |
500 |
100 |
500 |
2000 |
4000 |
6000 |
8000 |
12000 |
16000 |
20000 |
27500 |
35000 |
42500 |
|
|
Consumer quota rules |
500 |
100 |
500 |
2000 |
4000 |
6000 |
8000 |
12000 |
16000 |
20000 |
27500 |
35000 |
42500 |
|
|
K8s service sources |
1 |
N/A |
1 |
3 |
5 |
5 |
||||||||
|
Installed plugins |
N/A |
5 |
5 |
10 |
20 |
30 |
||||||||
|
Uploaded custom plugins |
N/A |
N/A |
N/A |
20 |
50 |
80 |
||||||||
For Serverless (Standard) and Serverless (Enterprise), purchasing a consumer rule expansion count simultaneously raises the limits of all three consumer-related quotas: Consumer authorization rules, Number of consumers, and Consumer quota rules. The maximum quota increase is subject to the purchase page. For billing information, see New Serverless billing overview. To request a quota increase for other gateway editions or other quota items of these Serverless editions, submit a ticket.
Consumer quota calculation
The limits for Number of consumers and Consumer authorization rules are listed separately by specification in the table above. See the instance quota table for other values.
Number of consumers = the number of distinct consumers referenced by direct authorizations and consumer-group authorizations on the current gateway. A consumer referenced by multiple rules or groups is counted once. An empty consumer group adds zero consumers.
Consumer authorization rules = the number of retained consumer authorization rules on the current gateway. Each direct authorization rule counts as one, and each consumer-group authorization rule also counts as one, regardless of the number of group members or whether the group is empty.
Each usage value is compared with its corresponding quota limit independently: the number of consumers must not exceed the Number of consumers limit, and the number of consumer authorization rules must not exceed the Consumer authorization rules limit. A whitelist or special configuration can raise the applicable limit, but does not change these counting rules.