Use the warm-up feature

Service Mesh (ASM) of Istio version 1.14 and later provides the warm-up feature to solve request timeout and data loss issues that occur when a new instance is starting. By using this feature, the number of requests sent to the new instance is progressively increased during the specified warm-up period. This topic describes how to use the warm-up feature.

Prerequisites

An ASM instance of the Enterprise Edition or Ultimate Edition is created, and the Istio version is 1.14.3 or later. For more information, see Create an ASM instance.
An ACK managed cluster is created. For more information, see Create an ACK managed cluster.
The cluster is added to the ASM instance. For more information, see Add a cluster to an ASM instance.
The Container Service for Kubernetes (ACK) cluster is connected by using kubectl. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
An ingress gateway service is created. For more information, see Create an ingress gateway service.
A sample application named Bookinfo is deployed. In this example, reviews v1 and reviews v2 are used. For more information, see Deploy an application in an ASM instance.

Background information

If the warm-up feature is disabled, a proportional amount of traffic is sent to the new pod when a pod is added. However, it takes time for some services to warm up before they can process all assigned requests. In this case, request timeout and data loss may occur, hampering user experience. For example, a web application running on a Java virtual machine (JVM) uses the horizontal pod autoscaling feature. After a pod is added, the pod is flooded with a large number of requests. Then, the requests are slowly responded or time out because the pod is warming up. The working principle of the warm-up feature is to allow a new pod to progressively receive traffic.

The warm-up feature allows you to specify a period of time for a service, during which the number of requests sent to every new pod progressively increases. When the specified period of time is over, the instance exits the warm-up mode.

The warm-up mode prevents new pods from being knocked down by a large number of requests. The new pods have time to warm up before processing requests that are distributed based on the specified load balancing policy.

This feature is helpful for applications that depend on cache and need to warm up to provide optimal performance. To enable this feature for a service, set the trafficPolicy.loadBalancer field in the YAML file of DestinationRule. Pay attention to the following fields:

loadBalancer: specifies the type of the load balancing policy. Valid values: ROUND_ROBIN and LEAST_REQUEST
warmupDurationSecs: specifies the warm-up duration of a service. A newly created endpoint of the service remains in the warm-up mode for the specified period of time after it is created. During this period of time, Istio progressively increases the amount of traffic for the endpoint instead of sending a proportional amount of traffic.

Sample DestinationRule YAML file:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: mocka
spec:
  host: mocka
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
      warmupDurationSecs: 100s
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

Step 1: Configure a routing rule and access the ingress gateway

In this example, the number of replicated pods for reviews v-1 and reviews-v2 Deployments in the ACK cluster is scaled to zero first.

Define the ingress gateway.

Create a file named bookinfo-gateway.yaml that contains the following content:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: bookinfo-gateway
spec:
  selector:
    istio: ingressgateway
  servers:
  - port:
      number: 80
      name: http
      protocol: HTTP
    hosts:
    - "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: bookinfo
spec:
  hosts:
  - "*"
  gateways:
  - bookinfo-gateway
  http:
  - match:
    - uri:
        exact: /productpage
    - uri:
        prefix: /static
    - uri:
        exact: /login
    - uri:
        exact: /logout
    - uri:
        prefix: /api/v1/products
    route:
    - destination:
        host: productpage
        port:
          number: 9080

Run the following command to deploy bookinfo-gateway:
```
kubectl apply -f bookinfo-gateway.yaml
```

Create a service named reviews.

Create a file named reviews.yaml that contains the following content:

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
    - labels:
        version: v1
      name: v1
    - labels:
        version: v2
      name: v2
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews
spec:
  hosts:
    - reviews
  http:
    - route:
        - destination:
            host: reviews

Run the following command to deploy the reviews service:
```
kubectl apply -f reviews.yaml
```

Continuously access the IP address of the ingress gateway and view the topology of your service mesh.
In this example, a hey command is used to send requests for 10 seconds to simulate stress scenarios. For more information about how to download and install hey, see hey. For more information about how to enable the observability feature of ASM and view the topology, see Use Mesh Topology to view the topology of an application.
```
hey -z 10s http://${IP address of the ingress gateway}/productpage
```
The following figure shows the call topology.

Step 2: View the number of requests sent to a new pod

Log on to the ASM console.
In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
In the left-side navigation pane, choose Observability Management Center > Monitoring indicators.
On the Monitoring indicators page, click the Monitoring instrument tab. On the Monitoring instrument tab, click the Cloud ASM Istio Service tab and select the reviews service.
Send requests for stress testing and observe monitoring data.
1. Make sure that the warm-up feature is disabled. Scale the number of replicated pods for the reviews-v2 Deployment in the ACK cluster from zero to one.
2. Run the following command to send requests to the ingress gateway for stress testing.
  In this example, the stress test lasts for 120 seconds.
```
hey -z 120s http://${IP address of the ingress gateway}/productpage
```
3. Observe the dashboard on the Prometheus Monitoring page.
  The pod of reviews-v2 starts receiving requests at a stable rate 15 seconds after the stress test begins. The amount of time required depends on the stress test environment.
Scale the number of replicated pods for the reviews-v2 Deployment in the ACK cluster to zero.

Step 3: Enable the warm-up feature

Update the reviews.yaml file to contain the following content:

Add the warmupDurationSecs field and set the value to 120s.

apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: reviews
spec:
  host: reviews
  subsets:
    - labels:
        version: v1
      name: v1
    - labels:
        version: v2
      name: v2
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
      warmupDurationSecs: 120s

Run the following command to update the reviews service:
```
kubectl apply -f reviews.yaml
```

Step 4: View the effect of the warm-up feature

Method 1: View the effect by using Prometheus

Scale the number of replicated pods for the reviews-v2 Deployment in the ACK cluster from zero to one.
Run the following command to send requests to the ingress gateway for stress testing.
In this example, the stress test lasts for 150 seconds.
```
hey -z 150s http://${IP address of the ingress gateway}/productpage
```
On the Cloud ASM Istio Service tab, view the dashboard for the service mesh.
The pod of reviews-v2 starts receiving requests at a stable rate 45 seconds after the stress test begins. The amount of time required depends on the stress test environment. Compared with Step 2, it takes more time because the amount of traffic sent to the new pod is progressively increased. When the warm-up feature is enabled, a new instance receives only a small number of requests when it is starting. The number of requests increases progressively during the specified period of time. When the specified period of time is over, the instance exits the warm-up mode.
When the warm-up feature is enabled, it takes about 150 seconds before traffic is evenly distributed to reviews-v1 and reviews-v2.

Method 2: View the effect by using the sidecar counter

Run the following command to clear the existing data of the counter.

This operation helps you view new data in the counter.

## the sidecar counter that corresponds to the pod of reviews-v1 
kubectl exec reviews-v1-55b668fc65-j****  -c istio-proxy -- curl localhost:15000/reset_counters -X POST

Run the following command to view the number of requests that are normally responded in the counter.

The value is 0 after the counter is cleared.

kubectl exec reviews-v1-55b668fc65-j****  -c istio-proxy -- curl localhost:15000/stats |grep inbound |grep upstream_rq_200

Expected output:

cluster.inbound|8000||.upstream_rq_200: 0

Change the number of replicated pods for the reviews-v2 Deployment in the ACK cluster and send requests to the ingress gateway for stress testing.
1. Scale the number of replicated pods for the reviews-v2 Deployment in the ACK cluster to zero and then to one.
2. Run the following command to send requests to the ingress gateway for 20 seconds:
```
hey -z 20s http://${IP address of the ingress gateway}/productpage
```
  Expected output:
```
Status code distribution:
  [200]    3260 responses
```
  The output shows that 3,260 requests were sent within 20 seconds by using the hey command.

Run the following command to view the number of requests that are normally responded in the counter that corresponds to reviews-v1:

kubectl exec reviews-v1-55b668fc65-j****  -c istio-proxy -- curl localhost:15000/stats |grep inbound |grep upstream_rq_200

Expected output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20873    0 20873    0     0  19.9M      0 --:--:-- --:--:-- --:--:-- 19.9M
cluster.inbound|9080||.external.upstream_rq_200: 2927
cluster.inbound|9080||.upstream_rq_200: 2927

Run the following command to view the number of requests that are normally responded in the counter that corresponds to reviews-v2:

kubectl exec reviews-v2-858f99c99-j****   -c istio-proxy -- curl localhost:15000/stats |grep inbound |grep upstream_rq_200

Expected output:

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 30149    0 30149    0     0  28.7M      0 --:--:-- --:--:-- --:--:-- 28.7M
cluster.inbound|9080||.external.upstream_rq_200: 333
cluster.inbound|9080||.upstream_rq_200: 333

The output shows that the pod of reviews-v2 processes only 10% of the requests in the first 20 seconds. The warm-up window is 120 seconds.

After the warm-up window ends, perform a stress test and check the warm-up effect.

Run the following commands to clear the counters for reviews-v1 and reviews-v2:

## the sidecar counter that corresponds to the pod of reviews-v1 
kubectl exec reviews-v1-55b668fc65-j****  -c istio-proxy -- curl localhost:15000/reset_counters -X POST
## the sidecar counter that corresponds to the pod of reviews-v2 
kubectl exec reviews-v2-858f99c99-j****  -c istio-proxy -- curl localhost:15000/reset_counters -X POST

Run the following command to send requests to the ingress gateway for 20 seconds:
```
hey -z 20s http://${IP address of the ingress gateway}/productpage
```

Run the following command to view the number of requests that are normally responded in the counter that corresponds to reviews-v1:

kubectl exec reviews-v1-55b668fc65-j****  -c istio-proxy -- curl localhost:15000/stats |grep inbound |grep upstream_rq_200

Expected output:

cluster.inbound|9080||.external.upstream_rq_200: 1600
cluster.inbound|9080||.upstream_rq_200: 1600

Run the following command to view the number of requests that are normally responded in the counter that corresponds to reviews-v2:
```
kubectl exec reviews-v2-858f99c99-j****   -c istio-proxy -- curl localhost:15000/stats |grep inbound |grep upstream_rq_200
```
Expected output:
```
cluster.inbound|9080||.external.upstream_rq_200: 1600
cluster.inbound|9080||.upstream_rq_200: 1600
```
The output shows that reviews-v1 and reviews-v2 receive and process the same number of requests after the warm-up window ends.