Use the local throttling feature of ASM - Alibaba Cloud Service Mesh

In scenarios such as flash sales, the traffic may instantaneously reach a peak that exceeds the maximum load supported by your system. As a result, a large number of calls are waiting to be processed, and the system stops responding. Service Mesh (ASM) provides the local throttling feature that you can use to throttle traffic for gateways and services. This way, you can protect your system from being overloaded. This topic describes how to use the local throttling feature of ASM.

Prerequisites

An ASM instance is created. The ASM instance meets the following requirements:
- If the ASM instance is of a commercial edition, the version of the ASM instance is V1.11.5.30 or later. If the version is earlier than V1.11.5.30, upgrade the ASM instance. For more information, see Update an ASM instance.
- If the ASM instance is of Standard Edition, the version of the ASM instance is V1.9 or later. In addition, you can use only the native rate limiting feature of Istio to implement local throttling for the ASM instance. The reference document varies with the Istio version. For more information about how to configure local throttling for the latest Istio version, see Enabling Rate Limits using Envoy.
The cluster is added to the ASM instance. For more information, see Add a cluster to an ASM instance.
An ingress gateway is deployed. For more information, see Create an ingress gateway.
The Bookinfo and NGINX services are created. For more information, see Deploy an application in an ASM instance. In this topic, the Bookinfo service is deployed in the default namespace, and the NGINX service is deployed in the foo namespace.

An ingress gateway is created. For more information, see Create an ingress gateway service.

Show the YAML code of the gateway

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: bookinfo-gateway
  namespace: default
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - bf2.example.com
    port:
      name: http
      number: 80
      protocol: http

A virtual service is created. For more information, see Manage virtual services.

Show the YAML code of the virtual service

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: bookinfo
  namespace: default
spec:
  gateways:
  - bookinfo-gateway
  hosts:
  - bf2.example.com
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        prefix: /static
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    name: productpage-route-name1
    route:
    - destination:
        host: productpage
        port:
          number: 9080
  - match:
    - uri:
        prefix: /nginx
    name: nginx-route-name1
    rewrite:
      uri: /
    route:
    - destination:
        host: nginx.foo.svc.cluster.local
        port:
          number: 80

The traffic generation tool hey is installed. For more information, visit hey at GitHub.

Scopes

The local throttling feature of ASM is applicable to ASM gateways and services for which sidecar proxies are injected.

Note

You can download the configuration files that are used in the following sample scenarios.

Sample scenario description

In the sample scenarios, the Bookinfo and NGINX services are used to describe how to throttle traffic for gateways and services. The NGINX service is separately deployed in the foo namespace to verify the scope in which throttling takes effect. 场景示例

Throttle traffic for a gateway

You can throttle traffic for a gateway that serves as a traffic ingress. This way, you can prevent downstream services from being overloaded.

Use kubectl to connect to the ASM instance. For more information, see Use kubectl on the control plane to access Istio resources. Create an ASMLocalRateLimiter by using the following configurations:

Note

In the configurations, the setting of the limite.quota field applies only to a single gateway instance. If the gateway has n instances, the threshold for the backend service corresponding to the test1 route is n × quota. If the number of backend service instances is changed, you must change the threshold accordingly.

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: for-api-test
  namespace: default
spec:
  workloadSelector:
    labels:
      app: istio-ingressgateway
  isGateway: true
  configs:
    - match:
        vhost:
          name: "www.example1.com" # If multiple vhosts are configured for the gateway, enter the name of the last vhost. 
          port: 80
          route:
            name_match: "test1" # The name of the route that is configured for the virtual service. If the virtual service does not have the specified route, the throttling does not take effect. 
      limit:
         fill_interval:
            seconds: 1
         quota: 10
    - match:
        vhost:
          name: "www.example2.com"
          port: 80
          route:
            name_match: "test1"
      limit:
         fill_interval:
            seconds: 1
         quota: 100

This section describes some of the fields. For the description of more fields, see Description of ASMCircuitBreaker fields.

workloadSelector: the one or more labels that specify a set of pods or VMs on which the throttling rule takes effect. In this example, the istio-ingressgateway label is specified so that the ASMLocalRateLimiter takes effect on a gateway.
isGateway: specifies whether the throttling rule takes effect on a gateway. In this example, the value is set to true.
seconds in fill_interval: the interval at which tokens are added to the bucket.
quota: the number of tokens that are added. In this topic, the seconds field is set to 1, and the quota field is set to 100. This indicates that a maximum of 100 tokens are added within 1 second. This way, the gateway can process up to 100 requests within 1 second.

Sample scenarios of gateway traffic throttling

Scenario 1: Configure a throttling rule for a single interface

Configure throttling for the productpage-route-name1 route of the bf2.example.com:80 vhost. This way, the access is throttled for the /productpage, /static, /login, and /logout interfaces that use the productpage-route-name1 route.

Use kubectl to connect to the ASM instance. For more information, see Use kubectl on the control plane to access Istio resources.

Create an ASMLocalRateLimiter.

Create a file named asmlocalratelimiter-test-gw.yaml and copy the following content to the file.

If the version of your ASM instances is earlier than 1.13.4, use the following content:

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      app: istio-ingressgateway
  isGateway: true
  configs:
  - limit:
      fill_interval:
        seconds: 1
      quota: 10
    match:
      vhost:
        name: bf2.example.com
        port: 80
        route:
          name_match: productpage-route-name1  # The name must be the same as that configured in the route configuration of the virtual service.

If the version of your ASM instances is 1.13.4 or later, use the following content:

You can specify the header and body content returned in the response by setting custom_response_body in the limit field.

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      app: istio-ingressgateway
  isGateway: true
  configs:
  - limit:
      fill_interval:
        seconds: 1
      quota: 10
      custom_response_body: '{"ret_code": xxx, "message": "Your request be limited" }'
    match:
      vhost:
        name: bf2.example.com
        port: 80
        route:
          name_match: productpage-route-name1  # The name must be the same as that configured in the route configuration of the virtual service.

Run the following command to create an ASMLocalRateLimiter:
```
kubectl apply -f asmlocalratelimiter-test-gw.yaml
```

Run the following commands in hey to generate continuous stress testing traffic:

hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ASM gateway>/productpage

hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ASM gateway>/nginx

Run the following command to access the /productpage interface of the Bookinfo service:

curl -H 'host: bf2.example.com'  http://<IP address of the ASM gateway>/productpage -v

Expected output:

< HTTP/1.1 429 Too Many Requests
< Content-Length: 18
< Content-Type: text/plain
< Date: Thu, 13 Jan 2022 03:03:09 GMT
< Server: istio-envoy
<
local_rate_limited

The access to the Bookinfo service is throttled.

Run the following command to access the /nginx interface of the Bookinfo service:
```
curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx  -v
```
The 429 error code is not found in the returned result. This indicates that the access is not throttled.

Scenario 2: Configure a global throttling rule for a vhost and port on a gateway

Configure throttling for the bf2.example.com:80 vhost. This way, the access is throttled for the /productpage, /static, /login, /logout, and /nginx interfaces of the Bookinfo service.

Use kubectl to connect to the ASM instance. For more information, see Use kubectl on the control plane to access Istio resources.

Create an ASMLocalRateLimiter.

Create a file named asmlocalratelimiter-test-gw-global.yaml and copy the following content to the file:

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      app: istio-ingressgateway
  isGateway: true
  configs:
    - match:
        vhost:
          name: "bf2.example.com"
          port: 80
      limit:
         fill_interval:
            seconds: 1
         quota: 10

Run the following command to create an ASMLocalRateLimiter:
```
kubectl apply -f asmlocalratelimiter-test-gw-global.yaml
```

Run the following command in hey to generate continuous stress testing traffic:
```
hey -host bf2.example.com -c 10 -n 100000 http://${ASM_GATEWAY_IP}/nginx
```
Run the following command to access the /nginx interface of the Bookinfo service:
```
curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx -v
```
The message HTTP/1.1 429 Too Many Requests is returned, which indicates that the access to the /nginx interface of the Bookinfo service is throttled.

Scenario 3: Delete the throttling configurations to stop throttling the access

Use kubectl to connect to the ASM instance. For more information, see Use kubectl on the control plane to access Istio resources.

Run the following command to delete the configuration files for throttling:

kubectl delete -f asmlocalratelimiter-test-gw.yaml

kubectl delete -f asmlocalratelimiter-test-gw-global.yaml

Run the following command to access the /nginx interface of the Bookinfo service:
```
curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx -v
```
The 429 error code is not found in the returned result. This indicates that the access is not throttled.

Throttle traffic for a service

You can throttle traffic for a service to limit the number of requests that can be processed by the service within a specific period of time.

Use kubectl to connect to the ASM instance. For more information, see Use kubectl on the control plane to access Istio resources. Create an ASMLocalRateLimiter by using the following configurations:

Note

The threshold that you specify applies only to a single service instance. If a service has n instances, the overall threshold for the service is n × quota_per_instance.

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: reviews-v3-local-ratelimiter
  namespace: default
spec:
  workloadSelector:
    labels:
      app: reviews
      version: v3
....

If a service has multiple versions, you can use a label in WorkLoadSelector to specify the deployment on which the throttling takes effect. For example, the preceding configurations indicate that the throttling takes effect only on the v3 version of the reviews service.

Sample scenarios of service traffic throttling

Important

Sidecar proxies must be injected for the services for which you want to configure throttling. Before you configure throttling for services, delete the throttling configurations that you configured for gateways to prevent the throttling of services from being affected.

Scenario 1: Configure a throttling rule for the reviews service

Use kubectl to connect to the ASM instance. For more information, see Use kubectl on the control plane to access Istio resources.

Create an ASMLocalRateLimiter.

Create a file named asmlocalratelimiter-test-reviews.yaml and copy the following content to the file:

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: reviews
  namespace: default
spec:
  workloadSelector:
    labels:
      app: reviews
  configs:
    - match:
        vhost:
          name: "*"
          port: 9080
          route:
            header_match:
            - name: ":path"
              prefix_match: "/"
      limit:
         fill_interval:
            seconds: 1
         quota: 10

The preceding configurations take effect on port 9080 of the reviews service. The header match conditions specified by the header_match field are used to match service requests in which the path starts with a forward slash (/).

Run the following command to create an ASMLocalRateLimiter:
```
kubectl apply -f asmlocalratelimiter-test-reviews.yaml
```

Run the following command in hey to generate continuous stress testing traffic:

hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ASM gateway>/productpage

Run the following command to view the sidecar logs for the productpage service:

kubectl logs -f productpage-v1-b84f8bfdd-wgxlc  -c istio-proxy

Expected output:

[2022-01-14T07:56:13.086Z] "GET /reviews/0 HTTP/1.1" 429 - via_upstream - "-" 0 18 1 1 "-" "hey/0.0.1" "9295da56-9a6b-9476-b662-1cbd61a82898" "reviews:9080" "10.180.0.190:9080" outbound|9080|v1|reviews.default.svc.cluster.local 10.180.0.196:36702 192.168.195.113:9080 10.180.0.196:33522 - -
[2022-01-14T07:56:13.091Z] "GET /reviews/0 HTTP/1.1" 429 - via_upstream - "-" 0 18 0 0 "-" "hey/0.0.1" "9295da56-9a6b-9476-b662-1cbd61a82898" "reviews:9080" "10.180.0.190:9080" outbound|9080|v1|reviews.default.svc.cluster.local 10.180.0.196:36702 192.168.195.113:9080 10.180.0.196:33528 - -
[2022-01-14T07:56:13.051Z] "GET /details/0 HTTP/1.1" 200 - via_upstream - "-" 0 178 41 1 "-" "hey/0.0.1" "061d3542-52a7-9511-b217-7fdf9ee9a1dd" "details:9080" "10.180.0.160:9080" outbound|9080||details.default.svc.cluster.local 10.180.0.196:58724 192.168.127.75:9080 10.180.0.196:57754 - default
[2022-01-14T07:56:13.095Z] "GET /reviews/0 HTTP/1.1" 429 - via_upstream - "-" 0 18 0 0 "-" "hey/0.0.1" "061d3542-52a7-9511-b217-7fdf9ee9a1dd" "reviews:9080" "10.180.0.190:9080" outbound|9080|v1|reviews.default.svc.cluster.local 10.180.0.196:36702 192.168.195.113:9080 10.180.0.196:33534 - -

The error code 429 is returned for requests to reviews:9080. This indicates that the access is throttled.

Scenario 2: Configure a throttling rule for the reviews service of the v3 version

Use kubectl to connect to the ASM instance. For more information, see Use kubectl on the control plane to access Istio resources.

Create an ASMLocalRateLimiter.

Create a file named asmlocalratelimiter-test-reviews-only-v3.yaml and copy the following content to the file:

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: reviews
  namespace: default
spec:
  workloadSelector:
    labels:
      app: reviews
      version: v3
  configs:
    - match:
        vhost:
          name: "*"
          port: 9080
          route:
            header_match:
            - name: ":path"
              prefix_match: "/"
      limit:
         fill_interval:
            seconds: 1
         quota: 10

The preceding configurations take effect on port 9080 of the reviews service of the v3 version. The header match conditions specified by the header_match field are used to match service requests in which the path starts with a forward slash (/).

Run the following command to create an ASMLocalRateLimiter:

kubectl apply -f asmlocalratelimiter-test-reviews-only-v3.yaml

Run the following commands in hey to generate continuous stress testing traffic:

hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ASM gateway>/productpage

Run the following command to view the sidecar logs for the pods of the reviews service of the v1, v2, and v3 versions:
```
kubectl logs -f ${your-reviews-pod-name}  -c istio-proxy
```
The error code 429 can be found only in the access logs for requests to the reviews service of the v3 version. The status code 200 is returned for all requests to the reviews service of the v1 and v2 versions. This indicates that only the access to the reviews service of the v3 version is throttled. The access to the reviews service of the v1 and v2 versions is not throttled.