Alibaba Cloud Service Mesh:Configure local throttling

Scenario 1: Apply throttling to a service port

This scenario shows how to apply local throttling to port 8000 of the httpbin service. Once configured, the rule applies to all requests sent to this specific port.

1. Create a local throttling rule.

   1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

   2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Traffic Protection and Scheduling. On the page that appears, click Create.

   3. On the Create page, configure the parameters and click OK.

      Section	Parameter	Description
      Basic Information About Throttling	Namespace	The namespace where the workload affected by local throttling resides. In this example, select default.
      	Name	A custom name for the local throttling configuration. In this example, enter httpbin.
      	Type of Effective Workload	The type of workload to which the throttling rule applies. Valid values: Application Service and Gateway. In this example, select Applicable Application.
      	Relevant Workload	Enter a set of label key-value pairs to select a specific workload. In this example, set Label Name to app and Label Value to httpbin.
      List of Throttling Rules	Service Port	The port number declared in the Kubernetes Service of the service. This must be an HTTP port. In this example, enter the HTTP port 8000 of the httpbin service.
      	Throttling Configuration	Specify the time window and the maximum number of requests allowed within that window for the token bucket algorithm. If the number of requests sent within the time window exceeds this limit, subsequent requests are throttled. This example uses the following configuration: Set Time Window for Throttling Detection to 60s. Set Allowed requests in window to 10. This configuration limits the service workload to a maximum of 10 requests in any 60-second window.

      The following YAML file is an example of the local throttling configuration.

      Local throttling configuration YAML

      apiVersion: istio.alibabacloud.com/v1beta1
      kind: ASMLocalRateLimiter
      metadata:
        name: httpbin
        namespace: default
      spec:
        configs:
          - limit:
              fill_interval:
                seconds: 60
              quota: 10
            match:
              vhost:
                name: '*'
                port: 8000
                route:
                  header_match:
                    - invert_match: false
                      name: ':path'
                      prefix_match: /
        isGateway: false
        workloadSelector:
          labels:
            app: httpbin

2. Verify the local throttling rule.

   1. Run the following command to start a bash session in the sleep container:

      kubectl exec -it deploy/sleep -- sh

   2. Run the following command to send 10 requests:

      for i in $(seq 1 10); do curl -v http://httpbin:8000/headers; done

   3. Run the following command to send an 11th request:

      curl -v http://httpbin:8000/headers

      Expected output:

      *   Trying 172.16.245.130:8000...
      * Connected to httpbin (172.16.245.130) port 8000
      > GET /headers HTTP/1.1
      > Host: httpbin:8000
      > User-Agent: curl/8.5.0
      > Accept: */*
      >
      < HTTP/1.1 429 Too Many Requests
      < x-local-rate-limit: true
      < content-length: 18
      < content-type: text/plain
      < date: Tue, 26 Dec 2023 08:02:58 GMT
      < server: envoy
      < x-envoy-upstream-service-time: 2

      The output shows an HTTP 429 status code, which indicates that the request was throttled.

Scenario 2: Apply throttling to a request path

This scenario shows how to apply local throttling to requests sent to the /headers path on port 8000 of the httpbin service. Once configured, the rule applies only to requests matching this specific path and port.

1. Create a local throttling rule.

   1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.

   2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Traffic Protection and Scheduling. On the page that appears, click Create.

   3. On the Create page, configure the parameters and click OK.

      Section	Parameter	Description
      Basic Information About Throttling	Namespace	The namespace where the workload affected by local throttling resides. In this example, select default.
      	Name	A custom name for the local throttling configuration. In this example, enter httpbin.
      	Type of Effective Workload	The type of workload to which the throttling rule applies. Valid values: Application Service and Gateway. In this example, select Applicable Application.
      	Relevant Workload	Enter a set of label key-value pairs to select a specific workload. In this example, set Label Name to app and Label Value to httpbin.
      List of Throttling Rules	Service Port	The port number declared in the Kubernetes Service of the service. This must be an HTTP port. In this example, enter the HTTP port 8000 of the httpbin service.
      	Match Request Attributes	Define the matching conditions for requests. The rule is applied only to requests that match these conditions. This example uses the following configuration: Set Matched Attributes to Request Path. Set Matching Method to Prefix Match. Set Matched Content to /headers.
      	Throttling Configuration	Specify the time window and the maximum number of requests allowed within that window for the token bucket algorithm. If the number of requests sent within the time window exceeds this limit, subsequent requests are throttled. This example uses the following configuration: Set Time Window for Throttling Detection to 60s. Set Allowed requests in window to 10. This configuration limits the service workload to a maximum of 10 requests in any 60-second window.

2. Verify the local throttling rule.

   1. Run the following command to start a bash session in the sleep container:

      kubectl exec -it deploy/sleep -- sh

   2. Run the following command to send 10 requests:

      for i in $(seq 1 10); do curl -v http://httpbin:8000/headers; done

   3. Run the following command to send an 11th request:

      curl -v http://httpbin:8000/headers

      Expected output:

      *   Trying 172.16.245.130:8000...
      * Connected to httpbin (172.16.245.130) port 8000
      > GET /headers HTTP/1.1
      > Host: httpbin:8000
      > User-Agent: curl/8.5.0
      > Accept: */*
      >
      < HTTP/1.1 429 Too Many Requests
      < x-local-rate-limit: true
      < content-length: 18
      < content-type: text/plain
      < date: Tue, 26 Dec 2023 08:02:58 GMT
      < server: envoy
      < x-envoy-upstream-service-time: 2

      The output shows an HTTP 429 status code, which indicates that the request was throttled.

   4. Run the following command to send a request to the /get path of the httpbin service:

      curl -v http://httpbin:8000/get

      Expected output:

      *   Trying 192.168.243.21:8000...
      * Connected to httpbin (192.168.243.21) port 8000 (#0)
      > GET /get HTTP/1.1
      > Host: httpbin:8000
      > User-Agent: curl/8.1.2
      > Accept: */*
      >
      < HTTP/1.1 200 OK
      < server: envoy
      < date: Thu, 11 Jan 2024 03:46:11 GMT
      < content-type: application/json
      < content-length: 431
      < access-control-allow-origin: *
      < access-control-allow-credentials: true
      < x-envoy-upstream-service-time: 1
      <
      {
        "args": {},
        "headers": {
          "Accept": "*/*",
          "Host": "httpbin:8000",
          "User-Agent": "curl/8.1.2",
          "X-Envoy-Attempt-Count": "1",
          "X-Forwarded-Client-Cert": "By=spiffe://cluster.local/ns/default/sa/httpbin;Hash=be10819991ba1a354a89e68b3bed1553c12a4fba8b65fbe0f16299d552680b29;Subject=\"\";URI=spiffe://cluster.local/ns/default/sa/sleep"
        },
        "origin": "127.0.0.6",
        "url": "http://httpbin:8000/get"
      }

      The output shows a 200 OK status code. This confirms that requests to other paths of the httpbin service are not affected by the throttling rule.

Related operations

FAQ

Why is my throttling rule not working?

Incorrect service protocol

Local throttling supports only the HTTP protocol. Before you configure a throttling rule, ensure that your service communicates over HTTP or an application-layer protocol built on HTTP, such as gRPC or Dubbo3.

In addition, you must define the service's protocol correctly so that the service mesh can accurately identify it. For more information, see How to correctly define the protocol type for a service.

Sidecar CRD impact on inbound traffic

By default, the service mesh automatically configures an inbound traffic listener for the sidecar proxy based on the port declarations in the Service definition. Both local throttling and global rate limiting rely on this default behavior.

You might use a Sidecar custom resource definition (CRD) to modify the service's default inbound traffic configuration, for example, to allow other pods to access an application that listens on localhost. For more information, see How to expose a localhost application to other pods.

In this case, because the default listener is modified, specifying the Kubernetes Service port in the local throttling rule will not work. You must instead set the Service Port in the local throttling rule to the actual inbound port specified in the Sidecar CRD.

For example, if you configure the following Sidecar CRD for the httpbin service while trying to apply local throttling to port 8000:

apiVersion: networking.istio.io/v1beta1
kind: Sidecar
metadata:
  name: localhost-access
  namespace: default
spec:
  ingress:
    - defaultEndpoint: '127.0.0.1:80'
      port:
        name: http
        number: 80
        protocol: HTTP
  workloadSelector:
    labels:
      app: httpbin

When you create the local throttling rule, you must set the Service Port to 80, not 8000.

Related documents

• If your ASM instance is version 1.19.0 or later, you can use the limit_overrides field in the ASMLocalRateLimiter YAML to match requests based on other attributes, such as query parameters. For more information, see ASMLocalRateLimiter CRD reference.

• Use the ASMGlobalRateLimiter CRD to configure global rate limiting for ingress gateways and application service traffic. For more information, see Configure global rate limiting for application services by using ASMGlobalRateLimiter.

• Configure local throttling or global rate limiting for an ingress gateway in the ASM console. For more information, see Configure local throttling for an ingress gateway and Configure global rate limiting for an ingress gateway.

• Use the warm-up feature to gradually increase the request volume over a specified period, which helps prevent issues like request timeouts and data loss. For more information, see Use the ASM warm-up feature.

• Configure a connection pool to implement circuit breaking, which protects your system from further damage during a failure or overload. For more information, see Configure a connection pool to implement circuit breaking.