Configure local throttling on an ingress gateway - Alibaba Cloud Service Mesh

In scenarios such as flash sales, the traffic may instantaneously reach a peak that exceeds the maximum load supported by your system. As a result, a large number of calls are waiting to be processed, and the system stops responding. Service Mesh (ASM) provides the local throttling feature that you can use to throttle traffic by using ingress gateways. This way, you can protect your system from being overloaded. This topic describes how to configure the local throttling feature on an ingress gateway.

Prerequisites

An ASM instance is created. The ASM instance meets the following requirements:
- If the ASM instance is of Enterprise Edition or Ultimate Edition, the version of the ASM instance must be v1.11.5.30 or later. If the version is earlier than v1.11.5.30, update the ASM instance. For more information, see Update an ASM instance.
- If the ASM instance is of Standard Edition, the version of the ASM instance must be v1.9 or later. In addition, you can use only the native rate limiting feature of Istio to implement local throttling for the ASM instance. The reference document varies with the Istio version. For more information about how to configure local throttling for the latest Istio version, see Enabling Rate Limits using Envoy.
The cluster is added to the ASM instance. For more information, see Add a cluster to an ASM instance.
An ingress gateway is deployed. For more information, see Create an ingress gateway.

The Bookinfo and NGINX services are created. In this topic, the Bookinfo service is deployed in the default namespace, and the NGINX service is deployed in the foo namespace. For more information about how to create the Bookinfo service, see Deploy an application in an ASM instance.

Expand to view how to create the NGINX service

Create an nginx.yaml file that contains the following content:

apiVersion: apps/v1 # for versions before 1.9.0 use apps/v1beta2
kind: Deployment
metadata:
  name: nginx
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx
        sidecarset-injected: "true"
    spec:
      containers:
      - name: nginx
        image: nginx:1.14.2
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: 80
  selector:
    app: nginx
  type: ClusterIP

Run the following command to deploy the NGINX service in the foo namespace:
```
kubectl apply -f nginx.yaml -n foo 
```

An Istio gateway and a virtual service are created. For more information, see Manage Istio gateways and Manage virtual services.

Expand to view the YAML code of the gateway

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: bookinfo-gateway
  namespace: default
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - bf2.example.com
    port:
      name: http
      number: 80
      protocol: http

Expand to view the YAML code of the virtual service

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: bookinfo
  namespace: default
spec:
  gateways:
  - bookinfo-gateway
  hosts:
  - bf2.example.com
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        prefix: /static
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    name: productpage-route-name1
    route:
    - destination:
        host: productpage
        port:
          number: 9080
  - match:
    - uri:
        prefix: /nginx
    name: nginx-route-name1
    rewrite:
      uri: /
    route:
    - destination:
        host: nginx.foo.svc.cluster.local
        port:
          number: 80

The traffic generation tool hey is installed. For more information, visit hey at GitHub.

Scopes

The local throttling feature of ASM is applicable to ingress gateways and services for which sidecar proxies are injected.

Note

You can download the configuration files that are used in the following sample scenarios.

Sample scenario description

In the sample scenarios, the Bookinfo and NGINX services are used to describe how to throttle traffic for ingress gateways and services. The NGINX service is separately deployed in the foo namespace to verify the scope in which throttling takes effect. 场景示例

Scenario 1: Configure a throttling rule for a route of the virtual service

Configure a throttling rule for the productpage-route-name1 route of the domain:port combination bf2.example.com:80. productpage-route-name1 is a route of the virtual service created for the Bookinfo service mentioned in the Prerequisites section. The productpage-route-name1 route is matched when a request is destined for one of the following paths: /productpage, /static, /login, and /logout. The request is then forwarded to the productpage service. After you configure a throttling rule, requests destined for the preceding paths are subject to traffic rate limiting.

Create a local throttling rule.

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Rate Limiting. On the page that appears, click Create.

On the Create page, configure the following parameters based on your business requirements and click OK.

For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.

Section	Parameter	Description
Basic Information About Throttling	Namespace	The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.
	Name	The name of the local throttling rule. For this example, enter ingressgateway.
	Type of Effective Workload	The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.
	Relevant Workload	If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.
List of Throttling Rules	Gateway Domain Name	The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.
	Gateway Port	The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.
	Match Routes of Virtual Service	The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. For this example, select productpage-route-name1.
	Throttling Configuration	Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example: Set Time Window for Throttling Detection to 1 second. Set Number of Requests Allowed in Time Window to 10. The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.
	Advanced Settings	Click Show Advanced Settings to configure some advanced settings when throttling is implemented. You can configure advanced settings based on your business requirements. For this example, enter `{"ret_code": xxx,"message": "Your request be limited"}` in the Custom Throttling Response Body field.

The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:

If you do not configure advanced settings, the default response is returned during throttling.

Expand to view the YAML code for local throttling when no advanced settings are configured

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  configs:
    - limit:
        fill_interval:
          seconds: 1
        quota: 10
      match:
        vhost:
          name: bf2.example.com
          port: 80
          route:
            name_match: productpage-route-name1
  isGateway: true
  workloadSelector:
    labels:
      istio: ingressgateway

If you have configured advanced settings, the custom response you specified in the advanced settings section is returned.

Expand to view the YAML code for local throttling when advanced settings are configured

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  configs:
    - limit:
        custom_response_body: '{"ret_code": xxx, "message": "Your request be limited" }'
        fill_interval:
          seconds: 1
        quota: 10
      match:
        vhost:
          name: bf2.example.com
          port: 80
          route:
            name_match: productpage-route-name1
  isGateway: true
  workloadSelector:
    labels:
      istio: ingressgateway

Run the following commands in hey to generate continuous stress testing traffic:

hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ingress gateway>/productpage

hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ingress gateway>/nginx

Run the following command to access the /productpage path of the ingress gateway:

curl -H 'host: bf2.example.com'  http://<IP address of the ingress gateway>/productpage -v

Expected output:

< HTTP/1.1 429 Too Many Requests
< Content-Length: 18
< Content-Type: text/plain
< Date: Thu, 13 Jan 2022 03:03:09 GMT
< Server: istio-envoy
<
local_rate_limited

The output indicates that the access to the Bookinfo service is throttled.

Run the following command to access the /nginx path of the ingress gateway:
```
curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx  -v
```
The 429 error code is not contained in the returned result. This indicates that the access is not throttled.

Scenario 2: Configure a throttling rule for the combination of a specific domain name and a specific port on an ingress gateway

Configure a throttling rule for the domain:port combination bf2.example.com:80 so that access to all paths of bf2.example.com:80 is throttled.

Configure a local throttling rule.

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Rate Limiting. On the page that appears, click Create.

On the Create page, configure the following parameters based on your business requirements and click OK.

For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.

Section	Parameter	Description
Basic Information About Throttling	Namespace	The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.
	Name	The name of the local throttling rule. For this example, enter ingressgateway.
	Type of Effective Workload	The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.
	Relevant Workload	If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.
List of Throttling Rules	Gateway Domain Name	The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.
	Gateway Port	The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.
	Match Routes of Virtual Service	The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. In this example, no route is selected. This indicates that the throttling rule takes effect on all routes destined for the specified domain name and port.
	Throttling Configuration	Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example: Set Time Window for Throttling Detection to 1 second. Set Number of Requests Allowed in Time Window to 10. The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.

The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:

Expand to view the YAML code for local throttling

apiVersion: istio.alibabacloud.com/v1beta1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  configs:
    - limit:
        fill_interval:
          seconds: 1
        quota: 10
      match:
        vhost:
          name: bf2.example.com
          port: 80
          route: {}
  isGateway: true
  workloadSelector:
    labels:
      istio: ingressgateway

Run the following command in hey to generate continuous stress testing traffic:
```
hey -host bf2.example.com -c 10 -n 100000 http://${ASM_GATEWAY_IP}/nginx
```
Run the following command to access the /nginx path of the ingress gateway:
```
curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx -v
```
The message HTTP/1.1 429 Too Many Requests is returned, which indicates that the access to the /nginx path of the ingress gateways is throttled.

Scenario 3: Configure a throttling rule for requests that contain specific headers for a single route of the virtual service

Configure a throttling rule for the nginx-route-name1 route of the domain:port combination bf2.example.com:80 and specify that the throttling rule takes effect only on requests with the ratelimit: "true" header. Other requests on the route are not subject to the throttling rule. nginx-route-name1 is a route of the virtual service created for the Bookinfo service mentioned in the Prerequisites section. The nginx-route-name1 route is matched when the request is destined for the /nginx. The request is then forwarded to the NGINX service.

Configure a local throttling rule.

Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Rate Limiting. On the page that appears, click Create.

On the Create page, configure the following parameters based on your business requirements and click OK.

For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.

Section	Parameter	Description
Basic Information About Throttling	Namespace	The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.
	Name	The name of the local throttling rule. For this example, enter ingressgateway.
	Type of Effective Workload	The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.
	Relevant Workload	If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.
List of Throttling Rules	Gateway Domain Name	The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.
	Gateway Port	The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.
	Match Routes of Virtual Service	The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. For this example, select nginx-route-name1.
	Match Request Attributes	The request matching rules. The configured throttling is triggered when requests meet the request matching rules. The following configurations are used in this example: In the Matched Attributes section, select Specific Request Header. Set Request Header to ratelimit. In the Matching Method section, select Exact Match. Enter true in the Matched Content field.
	Throttling Configuration	Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example: Set Time Window for Throttling Detection to 1 second. Set Number of Requests Allowed in Time Window to 10. The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.

The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:

Expand to view the YAML code for local throttling

apiVersion: istio.alibabacloud.com/v1
kind: ASMLocalRateLimiter
metadata:
  name: ingressgateway
  namespace: istio-system
spec:
  configs:
    - limit:
        fill_interval:
          seconds: 1
        quota: 10
      match:
        vhost:
          name: bf2.example.com
          port: 80
          route:
            header_match:
              - exact_match: 'true'
                invert_match: false
                name: ratelimit
            name_match: nginx-route-name1
  isGateway: true
  workloadSelector:
    labels:
      istio: ingressgateway

Run the following command in hey to initiate access by sending requests that contain the ratelimit: true header.
```
hey -host bf2.example.com -H 'ratelimit: true' -c 10 -n 10000 http://${ASM_GATEWAY_IP}/nginx
```
Run the following command to access the /nginx path of the ingress gateway:
```
curl -H 'host: bf2.example.com' -H 'ratelimit: true'  http://${ASM_GATEWAY_IP}/nginx -v
```
The HTTP/1.1 429 Too Many Requests message is returned. This indicates that access requests, which contain the ratelimit: true header, to the /nginx path of the ingress gateway are throttled.
Run the following command to access the /nginx path of the ingress gateway by sending requests that do not contain the ratelimit: true header:
```
curl -H 'host: bf2.example.com' http://${ASM_GATEWAY_IP}/nginx -v
```
The status code 429 is not contained in the response. This indicates that requests that do not contain the ratelimit: true header are not throttled.

Related operations

Delete the throttling configurations to stop access throttling

Delete a throttling rule.
1. Log on to the ASM console. In the left-side navigation pane, choose Service Mesh > Mesh Management.
2. On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose Traffic Management Center > Rate Limiting.
3. On the Rate Limiting page, find the throttling rule that you want to delete and click Delete in the Actions column. In the Submit message, click OK.
Run the following command to access the /nginx path of the ingress gateway:
```
curl -H 'host: bf2.example.com'  http://${ASM_GATEWAY_IP}/nginx -v
```
The 429 error code is not contained in the returned result. This indicates that the access is not throttled.

Alibaba Cloud Service Mesh:Configure local throttling on an ingress gateway

Prerequisites

Scopes

Sample scenario description

Scenario 1: Configure a throttling rule for a route of the virtual service

Scenario 2: Configure a throttling rule for the combination of a specific domain name and a specific port on an ingress gateway

Scenario 3: Configure a throttling rule for requests that contain specific headers for a single route of the virtual service

Related operations

Delete the throttling configurations to stop access throttling

References