The QuotaSchedulingPolicy CRD in the Service Mesh (ASM) traffic scheduling suite enables priority-based request scheduling under a specified call quota. When the in-flight request rate exceeds the quota, subsequent requests are queued and higher-priority requests are processed first.
Background information
QuotaSchedulingPolicy uses the token bucket algorithm to control the request rate for a specified service and queues requests when the rate exceeds the quota. It works as follows:
-
A rate limiter based on the token bucket algorithm limits the request rate. For more information about the algorithm, see the Background information section in Use RateLimitingPolicy to implement user-specific throttling.
-
When the request rate exceeds the quota, subsequent requests are queued and forwarded to the destination service after earlier requests complete. This keeps the request rate at the specified value. High-priority requests are dequeued and forwarded first.
Unlike throttling, QuotaSchedulingPolicy does not reject requests that exceed the quota. Instead, it places them in a priority queue and schedules them by priority while keeping the request rate within the quota.
Prerequisites
-
A Container Service for Kubernetes (ACK) managed cluster is added to your ASM instance, and the version of your ASM instance is V1.21.6.95 or later. For more information, see Add a cluster to an ASM instance.
-
You have connected to the ACK cluster by using kubectl. For more information, see Connect to an ACK cluster using kubectl.
-
The ASM traffic scheduling suite is enabled. For more information, see Enable the ASM traffic scheduling suite.
-
Automatic sidecar proxy injection is enabled for the default namespace in the ACK cluster. For more information, see Manage global namespaces.
-
An ASM ingress gateway named ingressgateway is created and port 80 is enabled. For more information, see Create an ingress gateway.
-
The HTTPBin application is deployed and can be accessed over a gateway. For more information, see Deploy the HTTPBin application.
Step 1: Create QuotaSchedulingPolicy
-
Use kubectl to connect to the ASM instance. For more information, see Access Istio resources with kubectl.
-
Create a quotaschedulingpolicy.yaml file that contains the following content:
apiVersion: istio.alibabacloud.com/v1 kind: QuotaSchedulingPolicy metadata: name: quotascheduling namespace: istio-system spec: quota_scheduler: bucket_capacity: 10 fill_amount: 10 rate_limiter: interval: 1s scheduler: workloads: - label_matcher: match_labels: http.request.header.user_type: guest parameters: priority: 50.0 name: guest - label_matcher: match_labels: http.request.header.user_type: subscriber parameters: priority: 200.0 name: subscriber selectors: - service: httpbin.default.svc.cluster.localThe following table describes some of the fields. For more information about the related fields, see QuotaSchedulingPolicy field reference.
Field
Description
fill_amount
The number of tokens added per interval. In this example, the value is 10, meaning 10 tokens are added to the bucket after each interval.
interval
The interval at which tokens are added to the bucket. In this example, the value is 1s, so 10 tokens are added every second.
bucket_capacity
The maximum number of tokens in the bucket. When the request rate is lower than the fill rate, tokens accumulate up to
bucket_capacity, allowing a degree of burst traffic. In this example, the value is 10, equal tofill_amount, so no burst traffic is allowed.workloads
Two types of requests are defined based on
user_typein the request headers:guestandsubscriber. The priority of a request of theguesttype is 50, and that of a request of thesubscribertype is 200.selectors
The services to which the quota policy applies. In this example, quota limiting is applied to the httpbin.default.svc.cluster.local service.
-
Run the following command to create QuotaSchedulingPolicy:
kubectl apply -f quotaschedulingpolicy.yamlStep 2: Verify whether QuotaSchedulingPolicy takes effect
In this example, the stress testing tool Fortio is used. For more information, see the Installation section ofFortio on the GitHub website.
-
Open two terminals and run the following stress testing commands simultaneously. In both tests, 10 concurrent connections send requests at 10,000 QPS, far exceeding the service's configured quota.
fortio load -c 10 -qps 10000 -H "user_type:guest" -t 30s -timeout 60s -a http://${IP address of the ASM ingress gateway}/status/201fortio load -c 10 -qps 10000 -H "user_type:subscriber" -t 30s -timeout 60s -a http://${IP address of the ASM ingress gateway}/status/202NoteReplace
${IP address of the ASM ingress gateway}in the preceding commands with the IP address of your ASM ingress gateway. For more information about how to obtain the IP address of the ASM ingress gateway, see substep 1 of Step 3 in the Version-based traffic routing with Istio topic.Expected output from test 1:
... # target 50% 4.83333 # target 75% 5.20763 # target 90% 5.38203 # target 99% 5.48668 # target 99.9% 5.49714 Sockets used: 10 (for perfect keepalive, would be 10) Uniform: false, Jitter: false Code 201 : 70 (100.0 %) Response Header Sizes : count 70 avg 249.94286 +/- 0.2871 min 248 max 250 sum 17496 Response Body/Total Sizes : count 70 avg 249.94286 +/- 0.2871 min 248 max 250 sum 17496 All done 70 calls (plus 10 warmup) 4566.839 ms avg, 2.1 qps Successfully wrote 4693 bytes of Json data to 2024-07-26-232250_114_55_5_155_status_201_iZbp1cz9ur77robaiv085tZ.jsonExpected output from test 2:
fortio load -c 10 -qps 10000 -H "user_type:subscriber" -t 30s -timeout 60s -a http://114.55.xx.xx/status/202 ... # target 50% 0.253333 # target 75% 1.875 # target 90% 4.26635 # target 99% 4.47301 # target 99.9% 4.49367 Sockets used: 10 (for perfect keepalive, would be 10) Uniform: false, Jitter: false Code 202 : 250 (100.0 %) Response Header Sizes : count 250 avg 250.264 +/- 0.4408 min 250 max 251 sum 62566 Response Body/Total Sizes : count 250 avg 250.264 +/- 0.4408 min 250 max 251 sum 62566 All done 250 calls (plus 10 warmup) 1226.657 ms avg, 8.0 qps Successfully wrote 4509 bytes of Json data to 2024-07-26-232250_114_55_5_155_status_202_iZbp1cz9ur77robaiv085tZ.jsonThe output shows that test 2 has about 1/4 the average latency and 4 times the QPS of test 1, because the subscriber priority (200) is four times the guest priority (50). A total of 320 requests are processed within 30 seconds across both tests. Excluding the 20 warm-up requests, the effective rate is exactly 10 requests per second, confirming that the service stays within the quota.
References
You can verify whether QuotaSchedulingPolicy takes effect in Grafana. Make sure the Prometheus instance for Grafana is configured with ASM traffic scheduling suite.
Import the following content into Grafana to create a QuotaSchedulingPolicy dashboard.
The dashboard is as follows:
