In scenarios such as flash sales, the traffic may instantaneously reach a peak that exceeds the maximum load supported by your system. As a result, a large number of calls are waiting to be processed, and the system stops responding. Service Mesh (ASM) provides the local throttling feature that you can use to throttle traffic by using ingress gateways. This way, you can protect your system from being overloaded. This topic describes how to configure the local throttling feature on an ingress gateway.
Prerequisites
An ASM instance is created. The ASM instance meets the following requirements:
If the ASM instance is of Enterprise Edition or Ultimate Edition, the version of the ASM instance must be v1.11.5.30 or later. If the version is earlier than v1.11.5.30, update the ASM instance. For more information, see Update an ASM instance.
If the ASM instance is of Standard Edition, the version of the ASM instance must be v1.9 or later. In addition, you can use only the native rate limiting feature of Istio to implement local throttling for the ASM instance. The reference document varies with the Istio version. For more information about how to configure local throttling for the latest Istio version, see Enabling Rate Limits using Envoy.
The cluster is added to the ASM instance. For more information, see Add a cluster to an ASM instance.
An ingress gateway is deployed. For more information, see Create an ingress gateway.
The Bookinfo and NGINX services are created. In this topic, the Bookinfo service is deployed in the default namespace, and the NGINX service is deployed in the foo namespace. For more information about how to create the Bookinfo service, see Deploy an application in an ASM instance.
An Istio gateway and a virtual service are created. For more information, see Manage Istio gateways and Manage virtual services.
The traffic generation tool hey is installed. For more information, visit hey at GitHub.
Scopes
The local throttling feature of ASM is applicable to ingress gateways and services for which sidecar proxies are injected.
You can download the configuration files that are used in the following sample scenarios.
Sample scenario description
In the sample scenarios, the Bookinfo and NGINX services are used to describe how to throttle traffic for ingress gateways and services. The NGINX service is separately deployed in the foo namespace to verify the scope in which throttling takes effect.
Scenario 1: Configure a throttling rule for a route of the virtual service
Configure a throttling rule for the productpage-route-name1 route of the domain:port combination bf2.example.com:80
. productpage-route-name1 is a route of the virtual service created for the Bookinfo service mentioned in the Prerequisites section. The productpage-route-name1 route is matched when a request is destined for one of the following paths: /productpage
, /static
, /login
, and /logout
. The request is then forwarded to the productpage service. After you configure a throttling rule, requests destined for the preceding paths are subject to traffic rate limiting.
Create a local throttling rule.
Log on to the ASM console. In the left-side navigation pane, choose .
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose . On the page that appears, click Create.
On the Create page, configure the following parameters based on your business requirements and click OK.
For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.
Section
Parameter
Description
Basic Information About Throttling
Namespace
The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.
Name
The name of the local throttling rule. For this example, enter ingressgateway.
Type of Effective Workload
The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.
Relevant Workload
If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.
List of Throttling Rules
Gateway Domain Name
The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.
Gateway Port
The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.
Match Routes of Virtual Service
The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. For this example, select productpage-route-name1.
Throttling Configuration
Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example:
Set Time Window for Throttling Detection to 1 second.
Set Number of Requests Allowed in Time Window to 10.
The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.
Advanced Settings
Click Show Advanced Settings to configure some advanced settings when throttling is implemented. You can configure advanced settings based on your business requirements. For this example, enter
{"ret_code": xxx,"message": "Your request be limited"}
in the Custom Throttling Response Body field.The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:
If you do not configure advanced settings, the default response is returned during throttling.
If you have configured advanced settings, the custom response you specified in the advanced settings section is returned.
Run the following commands in hey to generate continuous stress testing traffic:
hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ingress gateway>/productpage
hey -host bf2.example.com -c 10 -n 100000 http://<IP address of the ingress gateway>/nginx
Run the following command to access the /productpage path of the ingress gateway:
curl -H 'host: bf2.example.com' http://<IP address of the ingress gateway>/productpage -v
Expected output:
< HTTP/1.1 429 Too Many Requests < Content-Length: 18 < Content-Type: text/plain < Date: Thu, 13 Jan 2022 03:03:09 GMT < Server: istio-envoy < local_rate_limited
The output indicates that the access to the Bookinfo service is throttled.
Run the following command to access the /nginx path of the ingress gateway:
curl -H 'host: bf2.example.com' http://${ASM_GATEWAY_IP}/nginx -v
The 429 error code is not contained in the returned result. This indicates that the access is not throttled.
Scenario 2: Configure a throttling rule for the combination of a specific domain name and a specific port on an ingress gateway
Configure a throttling rule for the domain:port combination bf2.example.com:80
so that access to all paths of bf2.example.com:80 is throttled.
Configure a local throttling rule.
Log on to the ASM console. In the left-side navigation pane, choose .
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose . On the page that appears, click Create.
On the Create page, configure the following parameters based on your business requirements and click OK.
For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.
Section
Parameter
Description
Basic Information About Throttling
Namespace
The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.
Name
The name of the local throttling rule. For this example, enter ingressgateway.
Type of Effective Workload
The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.
Relevant Workload
If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.
List of Throttling Rules
Gateway Domain Name
The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.
Gateway Port
The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.
Match Routes of Virtual Service
The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. In this example, no route is selected. This indicates that the throttling rule takes effect on all routes destined for the specified domain name and port.
Throttling Configuration
Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example:
Set Time Window for Throttling Detection to 1 second.
Set Number of Requests Allowed in Time Window to 10.
The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.
The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:
Run the following command in hey to generate continuous stress testing traffic:
hey -host bf2.example.com -c 10 -n 100000 http://${ASM_GATEWAY_IP}/nginx
Run the following command to access the /nginx path of the ingress gateway:
curl -H 'host: bf2.example.com' http://${ASM_GATEWAY_IP}/nginx -v
The message
HTTP/1.1 429 Too Many Requests
is returned, which indicates that the access to the /nginx path of the ingress gateways is throttled.
Scenario 3: Configure a throttling rule for requests that contain specific headers for a single route of the virtual service
Configure a throttling rule for the nginx-route-name1 route of the domain:port combination bf2.example.com:80
and specify that the throttling rule takes effect only on requests with the ratelimit: "true"
header. Other requests on the route are not subject to the throttling rule. nginx-route-name1 is a route of the virtual service created for the Bookinfo service mentioned in the Prerequisites section. The nginx-route-name1 route is matched when the request is destined for the /nginx
. The request is then forwarded to the NGINX service.
Configure a local throttling rule.
Log on to the ASM console. In the left-side navigation pane, choose .
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose . On the page that appears, click Create.
On the Create page, configure the following parameters based on your business requirements and click OK.
For more information about the configuration of the parameters, see Description of ASMLocalRateLimiter fields.
Section
Parameter
Description
Basic Information About Throttling
Namespace
The namespace in which the workload for which the local throttling rule takes effect resides. In this example, you must select istio-system because all ingress gateways are deployed in the istio-system namespace.
Name
The name of the local throttling rule. For this example, enter ingressgateway.
Type of Effective Workload
The type of the workload for which throttling takes effect. You can select Applicable Application or Applicable Gateway. For this example, select Applicable Gateway.
Relevant Workload
If you select Applicable Gateway for Type of Effective Workload, you can select an Istio gateway to specify the workload in which the local throttling rule takes effect. This way, the workload in which the local throttling rule takes effect is the same as the workload selected for the Istio gateway. For this example, select bookinfo-gateway.
List of Throttling Rules
Gateway Domain Name
The domain name declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select bf2.example.com.
Gateway Port
The port declared in the Istio gateway. The specified port and domain name jointly determine where the throttling rule is applied. For this example, select 80.
Match Routes of Virtual Service
The route declared in the virtual service that defines the Istio gateway. The throttling rule takes effect for the specified route declared in the virtual service. For this example, select nginx-route-name1.
Match Request Attributes
The request matching rules. The configured throttling is triggered when requests meet the request matching rules. The following configurations are used in this example:
In the Matched Attributes section, select Specific Request Header.
Set Request Header to ratelimit.
In the Matching Method section, select Exact Match.
Enter true in the Matched Content field.
Throttling Configuration
Specifies the length of the time window for local throttling detection and the number of requests allowed in the time window. If the number of requests sent within the time window exceeds the upper limit, throttling is triggered for the requests. The following configurations are used in this example:
Set Time Window for Throttling Detection to 1 second.
Set Number of Requests Allowed in Time Window to 10.
The preceding configurations indicate that requests destined for workloads of this service cannot exceed 10 within 1 second.
The following YAML code shows the configurations of the local throttling rule specified in the preceding figure:
Run the following command in hey to initiate access by sending requests that contain the
ratelimit: true
header.hey -host bf2.example.com -H 'ratelimit: true' -c 10 -n 10000 http://${ASM_GATEWAY_IP}/nginx
Run the following command to access the
/nginx
path of the ingress gateway:curl -H 'host: bf2.example.com' -H 'ratelimit: true' http://${ASM_GATEWAY_IP}/nginx -v
The
HTTP/1.1 429 Too Many Requests
message is returned. This indicates that access requests, which contain theratelimit: true
header, to the/nginx
path of the ingress gateway are throttled.Run the following command to access the
/nginx
path of the ingress gateway by sending requests that do not contain theratelimit: true
header:curl -H 'host: bf2.example.com' http://${ASM_GATEWAY_IP}/nginx -v
The status code 429 is not contained in the response. This indicates that requests that do not contain the
ratelimit: true
header are not throttled.
Related operations
Delete the throttling configurations to stop access throttling
Delete a throttling rule.
Log on to the ASM console. In the left-side navigation pane, choose .
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose .
On the Rate Limiting page, find the throttling rule that you want to delete and click Delete in the Actions column. In the Submit message, click OK.
Run the following command to access the
/nginx
path of the ingress gateway:curl -H 'host: bf2.example.com' http://${ASM_GATEWAY_IP}/nginx -v
The 429 error code is not contained in the returned result. This indicates that the access is not throttled.