Configure local throttling for an ingress gateway - Alibaba Cloud Service Mesh

Sudden traffic spikes from events like flash sales can overwhelm backend services. Local throttling lets you limit traffic on the ingress gateway to protect your system.

How local throttling works

Local throttling uses a token bucket algorithm. The quota parameter sets the number of requests allowed within the fill_interval time window. When the bucket is empty, the gateway responds with 429 Too Many Requests until the bucket refills.

Parameter	Role in the token bucket	Example
`quota`	Bucket capacity -- maximum requests per interval	`10`
`fill_interval`	Refill period	`1 second`

Throttling granularity

The ASMLocalRateLimiter custom resource defines throttling rules. Each rule targets a gateway workload and specifies rate limits at one of the following levels:

Level	Behavior	Use case
Single route	Limits traffic matching a specific VirtualService route; other routes remain unaffected	Protect a specific backend service
Domain and port	Limits all traffic to a host:port pair, regardless of route	Apply a blanket rate limit across all services behind a domain
Route with header matching	Limits only requests on a specific route that carry a designated header; other requests pass through	Throttle specific clients or request types based on headers

Applicable scope

Local throttling applies to ASM gateways and application services with sidecar proxies injected.

Prerequisites

Before you begin, make sure that you have:

An ASM instance that meets the version requirements:
- Professional Edition or Ultimate Edition: version 1.11.5.30 or later. See Upgrade an ASM instance
- Standard Edition: version 1.9 or later. Standard Edition supports only the native Istio rate limiting feature. The reference document varies based on the Istio version. For the latest version, see Enabling Rate Limits using Envoy
A cluster added to the ASM instance
An ingress gateway deployed in the ASM instance
The Bookinfo service deployed in the default namespace and the Nginx service deployed in the foo namespace. See Deploy an application in a cluster associated with an ASM instance
A gateway rule and a virtual service configured for the sample services
The hey load testing tool installed

The following sections walk through all three throttling levels using the Bookinfo and Nginx sample services.

Create the Nginx service

Create an nginx.yaml file with the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
        sidecarset-injected: "true"
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 80
  selector:
    app: nginx
  type: ClusterIP

Deploy Nginx to the foo namespace:
```
kubectl apply -f nginx.yaml -n foo
```

Gateway YAML

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: bookinfo-gateway
  namespace: default
spec:
  selector:
 istio: ingressgateway
  servers:
  - hosts:
 - bf2.example.com
 port:
   name: http
   number: 80
   protocol: http

VirtualService YAML

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: bookinfo
  namespace: default
spec:
  gateways:
  - bookinfo-gateway
  hosts:
  - bf2.example.com
  http:
  - match:
 - uri:
     exact: /productpage
 - uri:
     prefix: /static
 - uri:
     exact: /login
 - uri:
     exact: /logout
 - uri:
     prefix: /api/v1/products
 name: productpage-route-name1
 route:
 - destination:
     host: productpage
     port:
       number: 9080
  - match:
 - uri:
     prefix: /nginx
 name: nginx-route-name1
 rewrite:
   uri: /
 route:
 - destination:
     host: nginx.foo.svc.cluster.local
     port:
       number: 80

Throttle a single virtual service route

This configuration throttles the productpage-route-name1 route for bf2.example.com:80. Requests matching /productpage, /static, /login, /logout, and /api/v1/products are rate-limited. The /nginx route remains unaffected.

Create the throttling rule

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Traffic Protection and Scheduling.

Click Create and configure the following parameters:

Section	Parameter	Value
Basic throttling information	Namespace	`istio-system` (all ASM gateways are deployed in this namespace)
	Name	`ingressgateway`
	Effective workload type	Apply to Gateway
	Associated workload	Select bookinfo-gateway under Select Gateway Rule
Throttling rule list	Gateway domain name	`bf2.example.com`
	Gateway port	`80`
	Match virtual service route	`productpage-route-name1`
Throttling configuration	Time Window For Throttling Detection	`1 second`
	Number of requests allowed in the time window	`10`

(Optional) Expand Show Advanced Settings to customize throttled responses. For example, enter {"ret_code": xxx,"message": "Your request be limited"} in Custom Throttling Response Body.
Click OK.

For parameter details, see ASMLocalRateLimiter CRD reference.

YAML without advanced options

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  configs:
 - limit:
     fill_interval:
       seconds: 1
     quota: 10
   match:
     vhost:
       name: bf2.example.com
       port: 80
       route:
         name_match: productpage-route-name1
  isGateway: true
  workloadSelector:
 labels:
   istio: ingressgateway

YAML with custom response body

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  configs:
 - limit:
     custom_response_body: '{"ret_code": xxx, "message": "Your request be limited" }'
     fill_interval:
       seconds: 1
     quota: 10
   match:
     vhost:
       name: bf2.example.com
       port: 80
       route:
         name_match: productpage-route-name1
  isGateway: true
  workloadSelector:
 labels:
   istio: ingressgateway

Verify that the route is throttled

Generate sustained load against both routes:

hey -host bf2.example.com -c 10 -n 100000 http://<ASM gateway IP address>/productpage
hey -host bf2.example.com -c 10 -n 100000 http://<ASM gateway IP address>/nginx

Send a request to the throttled /productpage path:

curl -H 'host: bf2.example.com' http://<ASM gateway IP address>/productpage -v

Expected output:

< HTTP/1.1 429 Too Many Requests
< Content-Length: 18
< Content-Type: text/plain
< Date: Thu, 13 Jan 2022 03:03:09 GMT
< Server: istio-envoy
<
local_rate_limited

The 429 Too Many Requests response confirms that the Bookinfo service is throttled.

Send a request to the /nginx path:
```
curl -H 'host: bf2.example.com' http://<ASM gateway IP address>/nginx -v
```
A normal response without a 429 status code confirms that the Nginx route is not affected.

Throttle all routes for a domain and port

This configuration throttles the entire bf2.example.com:80 combination. All routes under this domain and port -- including both /productpage and /nginx -- are rate-limited.

Create the throttling rule

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Traffic Protection and Scheduling.

Click Create and configure the following parameters:

Section	Parameter	Value
Basic throttling information	Namespace	`istio-system`
	Name	`ingressgateway`
	Effective workload type	Gateway
	Associated workload	Select bookinfo-gateway under Select Gateway Rule
Throttling rule list	Gateway domain name	`bf2.example.com`
	Gateway port	`80`
	Match virtual service route	Leave blank to match all routes
Throttling configuration	Time Window For Throttling Detection	`1 second`
	Number of requests allowed in the time window	`10`

Click OK.

YAML

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  configs:
 - limit:
     fill_interval:
       seconds: 1
     quota: 10
   match:
     vhost:
       name: bf2.example.com
       port: 80
       route: {}
  isGateway: true
  workloadSelector:
 labels:
   istio: ingressgateway

Verify that all routes are throttled

Generate sustained load:

hey -host bf2.example.com -c 10 -n 100000 http://<ASM gateway IP address>/nginx

Send a request to the /nginx path:
```
curl -H 'host: bf2.example.com' http://<ASM gateway IP address>/nginx -v
```
A 429 Too Many Requests response confirms that all routes under bf2.example.com:80 are throttled.

Throttle requests with a specific header on a single route

This configuration throttles the nginx-route-name1 route for bf2.example.com:80, but only for requests that carry the ratelimit: true header. Requests without this header pass through normally.

Create the throttling rule

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Traffic Protection and Scheduling.

Click Create and configure the following parameters:

Section	Parameter	Value
Basic throttling information	Namespace	`istio-system`
	Name	`ingressgateway`
	Effective workload type	Applicable Gateway
	Associated workload	Select bookinfo-gateway under Select Gateway Rule
Throttling rule list	Gateway domain name	`bf2.example.com`
	Gateway port	`80`
	Match virtual service route	`nginx-route-name1`
Match request properties	Match Attribute	Specific Request Header
	Request Header Name	`ratelimit`
	Matching Method	Exact Match
	Matched Content	`true`
Throttling configuration	Time Window For Throttling Detection	`1 second`
	Number of requests allowed in the time window	`10`

Click OK.

YAML

apiVersion: istio.alibabacloud.com/v1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  configs:
 - limit:
     fill_interval:
       seconds: 1
     quota: 10
   match:
     vhost:
       name: bf2.example.com
       port: 80
       route:
         header_match:
           - exact_match: 'true'
             invert_match: false
             name: ratelimit
         name_match: nginx-route-name1
  isGateway: true
  workloadSelector:
 labels:
   istio: ingressgateway

Verify that header-based throttling is applied

Generate sustained load with the ratelimit: true header:

hey -host bf2.example.com -H 'ratelimit: true' -c 10 -n 10000 http://<ASM gateway IP address>/nginx

Send a request with the header:
```
curl -H 'host: bf2.example.com' -H 'ratelimit: true' http://<ASM gateway IP address>/nginx -v
```
A 429 Too Many Requests response confirms that requests carrying the ratelimit: true header are throttled.
Send a request without the header:
```
curl -H 'host: bf2.example.com' http://<ASM gateway IP address>/nginx -v
```
A normal response without a 429 status code confirms that requests without the header are not affected.

Delete a throttling rule

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Traffic Protection and Scheduling.
Find the target throttling rule and click Delete in the Actions column. In the Confirm dialog box, click OK.
Verify that throttling is removed:
```
curl -H 'host: bf2.example.com' http://<ASM gateway IP address>/nginx -v
```
A normal response without a 429 status code confirms that the throttling rule has been removed.

Alibaba Cloud Service Mesh:Configure local throttling for an ingress gateway

How local throttling works

Throttling granularity

Applicable scope

Prerequisites

Throttle a single virtual service route

Create the throttling rule

Verify that the route is throttled

Throttle all routes for a domain and port

Create the throttling rule

Verify that all routes are throttled

Throttle requests with a specific header on a single route

Create the throttling rule

Verify that header-based throttling is applied

Delete a throttling rule

References