All Products
Search
Document Center

Alibaba Cloud Service Mesh:Configure local throttling on an ingress gateway

Last Updated:Mar 28, 2024

In scenarios such as flash sales, the traffic may instantaneously reach a peak that exceeds the maximum load supported by your system. As a result, a large number of calls are waiting to be processed, and the system stops responding. Service Mesh (ASM) provides the local throttling feature that you can use to throttle traffic by using ingress gateways. This way, you can protect your system from being overloaded. This topic describes how to configure the local throttling feature on an ingress gateway.

Prerequisites

  • An ASM instance is created. The ASM instance meets the following requirements:

    • If the ASM instance is of Enterprise Edition or Ultimate Edition, the version of the ASM instance must be v1.11.5.30 or later. If the version is earlier than v1.11.5.30, update the ASM instance. For more information, see Update an ASM instance.

    • If the ASM instance is of Standard Edition, the version of the ASM instance must be v1.9 or later. In addition, you can use only the native rate limiting feature of Istio to implement local throttling for the ASM instance. The reference document varies with the Istio version. For more information about how to configure local throttling for the latest Istio version, see Enabling Rate Limits using Envoy.

  • The cluster is added to the ASM instance. For more information, see Add a cluster to an ASM instance.

  • An ingress gateway is deployed. For more information, see Create an ingress gateway.

  • The Bookinfo and NGINX services are created. In this topic, the Bookinfo service is deployed in the default namespace, and the NGINX service is deployed in the foo namespace. For more information about how to create the Bookinfo service, see Deploy an application in an ASM instance.

    Expand to view how to create the NGINX service

    1. Create an nginx.yaml file that contains the following content:

      apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
      kind: Deployment
      metadata:
        name: nginx
      spec:
        selector:
          matchLabels:
            app: nginx
        replicas: 1
        template:
          metadata:
            labels:
              app: nginx
              sidecarset-injected: "true"
          spec:
            containers:
            - name: nginx
              image: nginx:1.14.2
              ports:
              - containerPort: 80
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: nginx
      spec:
        ports:
          - name: http
            port: 80
            protocol: TCP
            targetPort: 80
        selector:
          app: nginx
        type: ClusterIP
    2. Run the following command to deploy the NGINX service in the foo namespace:

      kubectl apply -f nginx.yaml -n foo 
  • An Istio gateway and a virtual service are created. For more information, see Manage Istio gateways and Manage virtual services.

    Expand to view the YAML code of the gateway

    apiVersion: networking.istio.io/v1beta1
    kind: Gateway
    metadata:
      name: bookinfo-gateway
      namespace: default
    spec:
      selector:
        istio: ingressgateway
      servers:
      - hosts:
        - bf2.example.com
        port:
          name: http
          number: 80
          protocol: http

    Expand to view the YAML code of the virtual service

    apiVersion: networking.istio.io/v1beta1
    kind: VirtualService
    metadata:
      name: bookinfo
      namespace: default
    spec:
      gateways:
      - bookinfo-gateway
      hosts:
      - bf2.example.com
      http:
      - match:
        - uri:
            exact: /productpage
        - uri:
            prefix: /static
        - uri:
            exact: /login
        - uri:
            exact: /logout
        - uri:
            prefix: /api/v1/products
        name: productpage-route-name1
        route:
        - destination:
            host: productpage
            port:
              number: 9080
      - match:
        - uri:
            prefix: /nginx
        name: nginx-route-name1
        rewrite:
          uri: /
        route:
        - destination:
            host: nginx.foo.svc.cluster.local
            port:
              number: 80
  • The traffic generation tool hey is installed. For more information, visit hey at GitHub.

Scopes

The local throttling feature of ASM is applicable to ingress gateways and services for which sidecar proxies are injected.

Note

You can download the configuration files that are used in the following sample scenarios.

Sample scenario description

In the sample scenarios, the Bookinfo and NGINX services are used to describe how to throttle traffic for ingress gateways and services. The NGINX service is separately deployed in the foo namespace to verify the scope in which throttling takes effect.场景示例

Scenario 1: Configure a throttling rule for a route of the virtual service

Configure a throttling rule for the productpage-route-name1 route of the domain:port combination bf2.example.com:80. productpage-route-name1 is a route of the virtual service created for the Bookinfo service mentioned in the Prerequisites section. The productpage-route-name1 route is matched when a request is destined for one of the following paths: /productpage, /static, /login, and /logout. The request is then forwarded to the productpage service. After you configure a throttling rule, requests destined for the preceding paths are subject to traffic rate limiting.

  1. Create a local throttling rule.

    1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

    2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Rate Limiting. On the page that appears, click Create.

    3. On the Create page, configure the following parameters based on your business requirements and click OK.

      For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.

      Section

      Parameter

      Description

      Basic Information About Throttling

      Namespace

      The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.

      Name

      The name of the local throttling rule. For this example, enter ingressgateway.

      Type of Effective Workload

      The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.

      Relevant Workload

      If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.

      List of Throttling Rules

      Gateway Domain Name

      The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.

      Gateway Port

      The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.

      Match Routes of Virtual Service

      The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. For this example, select productpage-route-name1.

      Throttling Configuration

      Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example:

      • Set Time Window for Throttling Detection to 1 second.

      • Set Number of Requests Allowed in Time Window to 10.

      The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.

      Advanced Settings

      Click Show Advanced Settings to configure some advanced settings when throttling is implemented. You can configure advanced settings based on your business requirements. For this example, enter {"ret_code": xxx,"message": "Your request be limited"} in the Custom Throttling Response Body field.

      image.png

      The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:

      • If you do not configure advanced settings, the default response is returned during throttling.

        Expand to view the YAML code for local throttling when no advanced settings are configured

        apiVersion: istio.alibabacloud.com/v1beta1
        kind: ASMLocalRateLimiter
        metadata:
          name: ingressgateway
          namespace: istio-system
        spec:
          configs:
            - limit:
                fill_interval:
                  seconds: 1
                quota: 10
              match:
                vhost:
                  name: bf2.example.com
                  port: 80
                  route:
                    name_match: productpage-route-name1
          isGateway: true
          workloadSelector:
            labels:
              istio: ingressgateway
      • If you have configured advanced settings, the custom response you specified in the advanced settings section is returned.

        Expand to view the YAML code for local throttling when advanced settings are configured

        apiVersion: istio.alibabacloud.com/v1beta1
        kind: ASMLocalRateLimiter
        metadata:
          name: ingressgateway
          namespace: istio-system
        spec:
          configs:
            - limit:
                custom_response_body: '{"ret_code": xxx, "message": "Your request be limited" }'
                fill_interval:
                  seconds: 1
                quota: 10
              match:
                vhost:
                  name: bf2.example.com
                  port: 80
                  route:
                    name_match: productpage-route-name1
          isGateway: true
          workloadSelector:
            labels:
              istio: ingressgateway
  2. Run the following commands in hey to generate continuous stress testing traffic:

    hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ingress gateway>/productpage
    hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ingress gateway>/nginx
  3. Run the following command to access the /productpage path of the ingress gateway:

    curl -H 'host: bf2.example.com'  http://<IP address of the ingress gateway>/productpage -v

    Expected output:

    < HTTP/1.1 429 Too Many Requests
    < Content-Length: 18
    < Content-Type: text/plain
    < Date: Thu, 13 Jan 2022 03:03:09 GMT
    < Server: istio-envoy
    <
    local_rate_limited

    The output indicates that the access to the Bookinfo service is throttled.

  4. Run the following command to access the /nginx path of the ingress gateway:

    curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx  -v

    The 429 error code is not contained in the returned result. This indicates that the access is not throttled.

Scenario 2: Configure a throttling rule for the combination of a specific domain name and a specific port on an ingress gateway

Configure a throttling rule for the domain:port combination bf2.example.com:80 so that access to all paths of bf2.example.com:80 is throttled.

  1. Configure a local throttling rule.

    1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

    2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Rate Limiting. On the page that appears, click Create.

    3. On the Create page, configure the following parameters based on your business requirements and click OK.

      For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.

      Section

      Parameter

      Description

      Basic Information About Throttling

      Namespace

      The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.

      Name

      The name of the local throttling rule. For this example, enter ingressgateway.

      Type of Effective Workload

      The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.

      Relevant Workload

      If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.

      List of Throttling Rules

      Gateway Domain Name

      The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.

      Gateway Port

      The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.

      Match Routes of Virtual Service

      The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. In this example, no route is selected. This indicates that the throttling rule takes effect on all routes destined for the specified domain name and port.

      Throttling Configuration

      Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example:

      • Set Time Window for Throttling Detection to 1 second.

      • Set Number of Requests Allowed in Time Window to 10.

      The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.

      image.png

      The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:

      Expand to view the YAML code for local throttling

      apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMLocalRateLimiter
      metadata:
        name: ingressgateway
        namespace: istio-system
      spec:
        configs:
          - limit:
              fill_interval:
                seconds: 1
              quota: 10
            match:
              vhost:
                name: bf2.example.com
                port: 80
                route: {}
        isGateway: true
        workloadSelector:
          labels:
            istio: ingressgateway
  2. Run the following command in hey to generate continuous stress testing traffic:

    hey -host bf2.example.com -c 10 -n 100000 http://${ASM_GATEWAY_IP}/nginx
  3. Run the following command to access the /nginx path of the ingress gateway:

    curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx -v

    The message HTTP/1.1 429 Too Many Requests is returned, which indicates that the access to the /nginx path of the ingress gateways is throttled.

Scenario 3: Configure a throttling rule for requests that contain specific headers for a single route of the virtual service

Configure a throttling rule for the nginx-route-name1 route of the domain:port combination bf2.example.com:80 and specify that the throttling rule takes effect only on requests with the ratelimit: "true" header. Other requests on the route are not subject to the throttling rule. nginx-route-name1 is a route of the virtual service created for the Bookinfo service mentioned in the Prerequisites section. The nginx-route-name1 route is matched when the request is destined for the /nginx. The request is then forwarded to the NGINX service.

  1. Configure a local throttling rule.

    1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

    2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Rate Limiting. On the page that appears, click Create.

    3. On the Create page, configure the following parameters based on your business requirements and click OK.

      For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.

      Section

      Parameter

      Description

      Basic Information About Throttling

      Namespace

      The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.

      Name

      The name of the local throttling rule. For this example, enter ingressgateway.

      Type of Effective Workload

      The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.

      Relevant Workload

      If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.

      List of Throttling Rules

      Gateway Domain Name

      The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.

      Gateway Port

      The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.

      Match Routes of Virtual Service

      The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. For this example, select nginx-route-name1.

      Match Request Attributes

      The request matching rules. The configured throttling is triggered when requests meet the request matching rules. The following configurations are used in this example:

      • In the Matched Attributes section, select Specific Request Header.

      • Set Request Header to ratelimit.

      • In the Matching Method section, select Exact Match.

      • Enter true in the Matched Content field.

      Throttling Configuration

      Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example:

      • Set Time Window for Throttling Detection to 1 second.

      • Set Number of Requests Allowed in Time Window to 10.

      The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.

      image.png

      The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:

      Expand to view the YAML code for local throttling

      apiVersion: istio.alibabacloud.com/v1
      kind: ASMLocalRateLimiter
      metadata:
        name: ingressgateway
        namespace: istio-system
      spec:
        configs:
          - limit:
              fill_interval:
                seconds: 1
              quota: 10
            match:
              vhost:
                name: bf2.example.com
                port: 80
                route:
                  header_match:
                    - exact_match: 'true'
                      invert_match: false
                      name: ratelimit
                  name_match: nginx-route-name1
        isGateway: true
        workloadSelector:
          labels:
            istio: ingressgateway
  2. Run the following command in hey to initiate access by sending requests that contain the ratelimit: true header.

    hey -host bf2.example.com -H 'ratelimit: true' -c 10 -n 10000 http://${ASM_GATEWAY_IP}/nginx
  3. Run the following command to access the /nginx path of the ingress gateway:

    curl -H 'host: bf2.example.com' -H 'ratelimit: true'  http://${ASM_GATEWAY_IP}/nginx -v

    The HTTP/1.1 429 Too Many Requests message is returned. This indicates that access requests, which contain the ratelimit: true header, to the /nginx path of the ingress gateway are throttled.

  4. Run the following command to access the /nginx path of the ingress gateway by sending requests that do not contain the ratelimit: true header:

    curl -H 'host: bf2.example.com' http://${ASM_GATEWAY_IP}/nginx -v

    The status code 429 is not contained in the response. This indicates that requests that do not contain the ratelimit: true header are not throttled.

Related operations

Delete the throttling configurations to stop access throttling

  1. Delete a throttling rule.

    1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

    2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Rate Limiting.

    3. On the Rate Limiting page, find the throttling rule that you want to delete and click Delete in the Actions column. In the Submit message, click OK.

  2. Run the following command to access the /nginx path of the ingress gateway:

    curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx -v

    The 429 error code is not contained in the returned result. This indicates that the access is not throttled.

References