In scenarios such as flash sales, the traffic may instantaneously reach a peak that exceeds the maximum load supported by your system. As a result, a large number of calls are waiting to be processed, and the system stops responding. Alibaba Cloud Service Mesh (ASM) provides the local throttling feature that you can use to throttle traffic for gateways and services. This way, you can protect your system from being overloaded. This topic describes how to use the local throttling feature of ASM.

Prerequisites

  • An ASM instance is created. The ASM instance meets the following requirements:
    • If the ASM instance is of a commercial edition, the version of the ASM instance must be V1.11.5.30 or later. For more information about how to update an ASM instance, see Update an ASM instance.
    • If the ASM instance is of Standard Edition, the version of the ASM instance must be V1.9 or later. In addition, you can use only the native rate limiting feature of Istio to implement local throttling for the ASM instance. The reference document varies with the Istio version. For more information about how to configure local throttling for the latest Istio version, see Enabling Rate Limits using Envoy.
  • The ACK cluster is added to the ASM instance. For more information, see Add a cluster to an ASM instance.
  • An ingress gateway service is deployed. For more information, see Deploy an ingress gateway service.
  • The Bookinfo and NGINX services are created. For more information, see Deploy an application in an ASM instance. In this topic, the Bookinfo service is deployed in the default namespace, and the NGINX service is deployed in the foo namespace.
  • A gateway is created for the ASM instance by using the following configurations. For more information, see Deploy an ingress gateway service. In this topic, the ASM gateway is deployed in the istio-system namespace.
    apiVersion: networking.istio.io/v1beta1
    kind: Gateway
    metadata:
      name: bookinfo-gateway
      namespace: default
    spec:
      selector:
        istio: ingressgateway
      servers:
      - hosts:
        - bf2.example.com
        port:
          name: http
          number: 80
          protocol: http
  • A virtual service is created by using the following configurations. For more information, see Manage virtual services.
    apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
      name: bookinfo
      namespace: default
    spec:
      gateways:
      - bookinfo-gateway
      hosts:
      - bf2.example.com
      http:
      - match:
        - uri:
            exact: /productpage
        - uri:
            prefix: /static
        - uri:
            exact: /login
        - uri:
            exact: /logout
        - uri:
            prefix: /api/v1/products
        name: productpage-route-name1
        route:
        - destination:
            host: productpage
            port:
              number: 9080
      - match:
        - uri:
            prefix: /nginx
        name: nginx-route-name1
        rewrite:
          uri: /
        route:
        - destination:
            host: nginx.foo.svc.cluster.local
            port:
              number: 80
  • The traffic generation tool hey is installed. For more information, visit hey at GitHub.

Declarative configurations of an ASMLocalRateLimiter

ASM allows you to declaratively define the configurations of an ASMLocalRateLimiter by using a custom resource definition (CRD). The following table describes the fields in the Spec section.

Field Type Description Required
workloadSelector map<string, string> The one or more labels that specify a set of pods or VMs on which the throttling rule takes effect. The scope of label search is restricted to the configuration namespace in which the resources reside. For more information, see Workload Selector. Yes
isGateway bool Specifies whether the throttling rule takes effect on a gateway. Default value: false. No
configs LocalRateLimiterConfig[] The local throttling rule. You can configure multiple local throttling rules. Yes

Attribute fields of LocalRateLimiterConfig

Field Type Description Required
name string The name of the throttling rule. No
match RatelimitMatch The match conditions. No
limit LimitConfig The threshold configurations. No
Attribute fields of RatelimitMatch
Field Type Description Required
vhost VirtualHostMatch The match conditions for the vhost. No

Attribute fields of LimitConfig

Field Type Description Required
fill_interval Duration The interval of issuing tokens. Examples: seconds: 1 and nanos: 1000. nanos indicates nanoseconds. No
quota int The number of tokens. The value must be an integer. Example: 1000. No

Attribute fields of VirtualHostMatch

Field Type Description Required
name string The name of the matched vhost. No
port int The matched request port. No
route RouteMatch The name of the route corresponding to the matched request port. No

Attribute fields RouteMatch

Field Type Description Required
name_match string The name of the matched route. This parameter specifies a single route in a virtual service. No
header_match HeaderMatcher[] The header for matching service requests. You can configure multiple headers. No

Attribute fields of HeaderMatcher

Field Type Description Required
name string The name of the header. No
regex_match string The value that is used for regular expression match. No
exact_match string The value that is used for exact match. No
prefix_match string The value that is used for prefix match. No
suffix_match string The value that is used for suffix match. No
present_match bool Specifies whether to check only the availability of the specified header. If you set this parameter to true, the system only checks whether the header exists regardless of the value of the header. No
invert_match bool Specifies whether to invert the result of regular expression match. Default value: false. If you set this parameter to true, the result of regular expression match is inverted. No

Application scope

The local throttling feature of ASM is applicable to ASM gateways and services for which sidecar proxies are injected.
Note You can download the configuration files that are used in the following sample scenarios.

Sample scenario description

In the sample scenarios, the Bookinfo and NGINX services are used to describe how to throttle traffic for gateways and services. The NGINX service is separately deployed in the foo namespace to verify the scope in which throttling takes effect. Sample scenarios

Throttle traffic for a gateway

You can throttle traffic for a gateway that serves as a traffic ingress. This way, you can prevent downstream services from being overloaded.

Use kubectl to connect to the ASM instance. For more information, see Use kubectl to connect to an ASM instance. Create an ASMLocalRateLimiter by using the following configurations:
Notice In the configurations, the setting of the limit.quota field applies only to a single gateway instance. If the gateway has n instances, the threshold for the backend service corresponding to the test1 route is n × quota. If the number of backend service instances is changed, you must change the threshold accordingly.
apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: for-api-test
  namespace: default
spec:
  workloadSelector:
    labels:
      app: istio-ingressgateway
  isGateway: true
  configs:
    - match:
        vhost:
          name: "www.example1.com"   ## If multiple vhosts are configured for the gateway, enter the name of the last vhost. 
          port: 80
          route:
            name_match: "test1"   ## The name of the route that is configured for the virtual service. If the virtual service does not have the specified route, the throttling does not take effect. 
      limit:
         fill_interval:
            seconds: 1
         quota: 10
    - match:
        vhost:
          name: "www.example2.com"
          port: 80
          route:
            name_match: "test1"
      limit:
         fill_interval:
            seconds: 1
         quota: 100
  • workloadSelector: the one or more labels that specify a set of pods or VMs on which the throttling rule takes effect. In this example, the app: istio-ingressgateway label is specified so that the ASMLocalRateLimiter takes effect on a gateway.
  • isGateway: specifies whether the throttling rule takes effect on a gateway. In this example, the value is set to true.
  • seconds in fill_interval: the interval of issuing tokens.
  • quota: the number of tokens that are issued.

    In this topic, the seconds field is set to 1, and the quota field is set to 100. This indicates that 100 tokens are issued within 1 second. This way, the gateway can process up to 100 requests within 1 second.

Sample scenarios of gateway traffic throttling

Scenario 1: Configure a throttling rule for a single interface

Configure throttling for the productpage-route-name1 route of the vhost bf2.example.com:80. This way, the access is throttled for the /productpage, /static, /login, and /logout interfaces that use the productpage-route-name1 route.

  1. Use kubectl to connect to the ASM instance. For more information, see Use kubectl to connect to an ASM instance.
  2. Create an ASMLocalRateLimiter.
    1. Create the asmlocalratelimiter-test-gw.yaml file that contains the following content:
      apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMLocalRateLimiter
      metadata:
        name: ingressgateway
        namespace: istio-system
      spec:
        workloadSelector:
          labels:
            app: istio-ingressgateway
        isGateway: true
        configs:
        - limit:
            fill_interval:
              seconds: 1
            quota: 10
          match:
            vhost:
              name: bf2.example.com
              port: 80
              route:
                name_match: productpage-route-name1  ## The name must be the same as that configured in the route configuration of the virtual service. 
    2. Run the following command to create an ASMLocalRateLimiter:
      kubectl apply -f asmlocalratelimiter-test-gw.yaml
  3. Run the following commands in hey to generate continuous pressure traffic:
    hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ASM gateway>/productpage
    hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ASM gateway>/nginx
  4. Run the following command to access the /productpage interface of the Bookinfo service:
    curl -H 'host: bf2.example.com'  http://<IP address of the ASM gateway>/productpage -v

    Expected output:

    < HTTP/1.1 429 Too Many Requests
    < Content-Length: 18
    < Content-Type: text/plain
    < Date: Thu, 13 Jan 2022 03:03:09 GMT
    < Server: istio-envoy
    <
    local_rate_limited

    The access to the Bookinfo service is throttled.

  5. Run the following command to access the /nginx interface of the Bookinfo service:
    curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx  -v

    The 429 error code is not found in the returned result. This indicates that the access is not throttled.

Scenario 2: Configure a global throttling rule for a vhost and port on a gateway

Configure throttling for the vhost bf2.example.com:80. This way, the access is throttled for the /productpage, /static, /login, /logout, and /nginx interfaces of the Bookinfo service.

  1. Use kubectl to connect to the ASM instance. For more information, see Use kubectl to connect to an ASM instance.
  2. Create an ASMLocalRateLimiter.
    1. Create the asmlocalratelimiter-test-gw-global.yaml file that contains the following content:
      apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMLocalRateLimiter
      metadata:
        name: ingressgateway
        namespace: istio-system
      spec:
        workloadSelector:
          labels:
            app: istio-ingressgateway
        isGateway: true
        configs:
          - match:
              vhost:
                name: "bf2.example.com"
                port: 80
            limit:
               fill_interval:
                  seconds: 1
               quota: 10
    2. Run the following command to create an ASMLocalRateLimiter:
      kubectl apply -f asmlocalratelimiter-test-gw-global.yaml
  3. Run the following command in hey to generate continuous pressure traffic:
    hey -host bf2.example.com -c 10 -n 100000 http://${ASM_GATEWAY_IP}/nginx 
  4. Run the following command to access the /nginx interface of the Bookinfo service:
    curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx -v

    The message HTTP/1.1 429 Too Many Requests is returned, indicating that the access to the /nginx interface of the Bookinfo service is throttled.

Scenario 3: Delete the throttling configurations to stop throttling the access

  1. Use kubectl to connect to the ASM instance. For more information, see Use kubectl to connect to an ASM instance.
  2. Run the following commands to delete the configuration files for throttling:
    kubectl delete -f asmlocalratelimiter-test-gw.yaml
    kubectl delete -f asmlocalratelimiter-test-gw-global.yaml
  3. Run the following command to access the /nginx interface of the Bookinfo service:
    curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx -v

    The 429 error code is not found in the returned result. This indicates that the access is not throttled.

Throttle traffic for a service

You can throttle traffic for a service to limit the number of requests that can be processed by the service within a specific period of time.

Use kubectl to connect to the ASM instance. For more information, see Use kubectl to connect to an ASM instance. Create an ASMLocalRateLimiter by using the following configurations:
Note The threshold that you specify applies only to a single service instance. If a service has n instances, the overall threshold for the service is n × quota_per_instance.
apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: reviews-v3-local-ratelimiter
  namespace: default
spec:
  workloadSelector:
    labels:
      app: reviews
      version: v3
....

If a service has multiple versions, you can use a label in workloadSelector to specify the deployment on which the throttling takes effect. For example, the preceding configurations indicate that the throttling takes effect only on the v3 version of the reviews service.

Sample scenarios of service traffic throttling

Note Sidecar proxies must be injected for the services for which you want to configure throttling. Before you configure throttling for services, delete the throttling configurations that you configure for gateways to prevent the throttling of services from being affected.

Scenario 1: Configure a throttling rule for the reviews service

  1. Use kubectl to connect to the ASM instance. For more information, see Use kubectl to connect to an ASM instance.
  2. Create an ASMLocalRateLimiter.
    1. Create the asmlocalratelimiter-test-reviews.yaml file that contains the following content:
      apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMLocalRateLimiter
      metadata:
        name: reviews
        namespace: default
      spec:
        workloadSelector:
          labels:
            app: reviews
        configs:
          - match:
              vhost:
                name: "*"
                port: 9080
                route:
                  header_match:
                  - name: ":path"
                    prefix_match: "/"
            limit:
               fill_interval:
                  seconds: 1
               quota: 10

      The preceding configurations take effect on port 9080 of the reviews service. The header match conditions specified by the header_match field are used to match service requests in which the path starts with a forward slash (/).

    2. Run the following command to create an ASMLocalRateLimiter:
      kubectl apply -f asmlocalratelimiter-test-reviews.yaml
  3. Run the following command in hey to generate continuous pressure traffic:
    hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ASM gateway>/productpage
  4. Run the following command to view the sidecar logs for the productpage service:
    kubectl logs -f productpage-v1-b84f8bfdd-wgxlc  -c istio-proxy

    Expected output:

    
    [2022-01-14T07:56:13.086Z] "GET /reviews/0 HTTP/1.1" 429 - via_upstream - "-" 0 18 1 1 "-" "hey/0.0.1" "9295da56-9a6b-9476-b662-1cbd61a82898" "reviews:9080" "10.180.0.190:9080" outbound9080v1reviews.default.svc.cluster.local 10.180.0.196:36702 192.168.195.113:9080 10.180.0.196:33522 - -
    [2022-01-14T07:56:13.091Z] "GET /reviews/0 HTTP/1.1" 429 - via_upstream - "-" 0 18 0 0 "-" "hey/0.0.1" "9295da56-9a6b-9476-b662-1cbd61a82898" "reviews:9080" "10.180.0.190:9080" outbound9080v1reviews.default.svc.cluster.local 10.180.0.196:36702 192.168.195.113:9080 10.180.0.196:33528 - -
    [2022-01-14T07:56:13.051Z] "GET /details/0 HTTP/1.1" 200 - via_upstream - "-" 0 178 41 1 "-" "hey/0.0.1" "061d3542-52a7-9511-b217-7fdf9ee9a1dd" "details:9080" "10.180.0.160:9080" outbound9080details.default.svc.cluster.local 10.180.0.196:58724 192.168.127.75:9080 10.180.0.196:57754 - default
    [2022-01-14T07:56:13.095Z] "GET /reviews/0 HTTP/1.1" 429 - via_upstream - "-" 0 18 0 0 "-" "hey/0.0.1" "061d3542-52a7-9511-b217-7fdf9ee9a1dd" "reviews:9080" "10.180.0.190:9080" outbound9080v1reviews.default.svc.cluster.local 10.180.0.196:36702 192.168.195.113:9080 10.180.0.196:33534 - -

    The error code 429 is returned for requests to reviews:9080. This indicates that the access is throttled.

Scenario 2: Configure a throttling rule for the reviews service of the v3 version

  1. Use kubectl to connect to the ASM instance. For more information, see Use kubectl to connect to an ASM instance.
  2. Create an ASMLocalRateLimiter.
    1. Create the asmlocalratelimiter-test-reviews-only-v3.yaml file that contains the following content:
      apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMLocalRateLimiter
      metadata:
        name: reviews
        namespace: default
      spec:
        workloadSelector:
          labels:
            app: reviews
            version: v3
        configs:
          - match:
              vhost:
                name: "*"
                port: 9080
                route:
                  header_match:
                  - name: ":path"
                    prefix_match: "/"
            limit:
               fill_interval:
                  seconds: 1
               quota: 10

      The preceding configurations take effect on port 9080 of the reviews service of the v3 version. The header match conditions specified by the header_match field are used to match service requests in which the path starts with a forward slash (/).

    2. Run the following command to create an ASMLocalRateLimiter:
      kubectl apply -f asmlocalratelimiter-test-reviews-only-v3.yaml
  3. Run the following command in hey to generate continuous pressure traffic:
    hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ASM gateway>/productpage
  4. Run the following command to view the sidecar logs for the pods of the reviews service of the v1, v2, and v3 versions:
    kubectl logs -f ${your-reviews-pod-name}  -c istio-proxy

    The error code 429 can be found only in the access logs for requests to the reviews service of the v3 version. The status code 200 is returned for all requests to the reviews service of the v1 and v2 versions. This indicates that only the access to the reviews service of the v3 version is throttled. The access to the reviews service of the v1 and v2 versions is not throttled.