Use MSE to achieve graceful start and shutdown for microservice applications and avoid traffic loss - Microservices Engine

To avoid traffic loss during deployment, many high-concurrency applications deploy only during off-peak hours. Although this approach helps reduce issues, it still incurs high operational costs due to unpredictable factors. MSE deeply optimizes application startup and shutdown processes. During shutdown, MSE uses adaptive waiting and proactive notification mechanisms to ensure all pending requests complete before the service goes offline, preventing abrupt service interruptions. During startup, MSE aligns microservice lifecycle management precisely with each deployment phase using readiness probes, ensuring stable launches of new service versions.

Prerequisites

Demo architecture overview

Assume the application architecture consists of a Zuul gateway and backend microservice application instances (Spring Cloud). The backend call chain includes shopping cart application A, transaction center application B, and inventory center application C. These applications use Nacos for service registration and discovery.

In the spring-cloud-zuul application, as shown in the following figure, it simultaneously calls both the canary release version and the normal version of spring-cloud-a at a rate of 100 queries per second (QPS). 应用部署流量架构图

Deploy applications and enable microservice governance

Important

Because the demo includes CronHPA tasks, first install the ack-kubernetes-cronhpa-controller component in your cluster. For details, see Step 1: Install the CronHPA component.

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, click the name of your cluster. In the left navigation pane, click Workloads > Deployments.
On the Deployments page, click Create from YAML.

For Sample Template, select Custom. Paste the following YAML into the Template field, then click Create.

This demo file is named mse-demo.yaml. It deploys the Zuul gateway and three applications (A, B, and C). Applications A and B each have a baseline version and a canary release version. The baseline version of application B has graceful shutdown disabled, while its canary version has graceful shutdown enabled. Application C has service warmup enabled with a warmup duration of 120 seconds.

Expand to view the YAML file

# Nacos Server
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nacos-server
  name: nacos-server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nacos-server
  template:
    metadata:
      labels:
        app: nacos-server
        msePilotCreateAppName: nacos-server
        msePilotAutoEnable: "on"
    spec:
      containers:
      - env:
        - name: MODE
          value: standalone
        image: registry.cn-shanghai.aliyuncs.com/yizhan/nacos-server:latest
        imagePullPolicy: Always
        name: nacos-server
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
      dnsPolicy: ClusterFirst
      restartPolicy: Always
# Nacos Server Service configuration
---
apiVersion: v1
kind: Service
metadata:
  name: nacos-server
spec:
  ports:
  - port: 8848
    protocol: TCP
    targetPort: 8848
  selector:
    app: nacos-server
  type: ClusterIP
# Zuul gateway application
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: spring-cloud-zuul
spec:
  replicas: 1
  selector:
    matchLabels:
      app: spring-cloud-zuul
  template:
    metadata:
      labels:
        app: spring-cloud-zuul
        msePilotCreateAppName: spring-cloud-zuul
        msePilotAutoEnable: "on"
    spec:
      containers:
        - env:
            - name: JAVA_HOME
              value: /usr/lib/jvm/java-1.8-openjdk/jre
            - name: LANG
              value: C.UTF-8
          image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-zuul:1.0.1
          imagePullPolicy: Always
          name: spring-cloud-zuul
          ports:
            - containerPort: 20000
# Application A baseline version, with full-link pass-through enabled by machine dimension.
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: spring-cloud-a
  name: spring-cloud-a
spec:
  replicas: 2
  selector:
    matchLabels:
      app: spring-cloud-a
  template:
    metadata: 
      labels:
        app: spring-cloud-a
        msePilotCreateAppName: spring-cloud-a
        msePilotAutoEnable: "on"
    spec:
      containers:
      - env:
        - name: LANG
          value: C.UTF-8
        - name: JAVA_HOME
          value: /usr/lib/jvm/java-1.8-openjdk/jre
        - name: profiler.micro.service.tag.trace.enable
          value: "true"
        image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-a:0.1-SNAPSHOT
        imagePullPolicy: Always
        name: spring-cloud-a
        ports:
        - containerPort: 20001
          protocol: TCP
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
        livenessProbe:
          tcpSocket:
            port: 20001
          initialDelaySeconds: 10
          periodSeconds: 30
# Application A canary version, with full-link pass-through enabled by machine dimension.
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: spring-cloud-a-gray
  name: spring-cloud-a-gray
spec:
  replicas: 2
  selector:
    matchLabels:
      app: spring-cloud-a-gray
  strategy:
  template:
    metadata:
      labels:
        alicloud.service.tag: gray
        app: spring-cloud-a-gray
        msePilotCreateAppName: spring-cloud-a
        msePilotAutoEnable: "on"
    spec:
      containers:
      - env:
        - name: LANG
          value: C.UTF-8
        - name: JAVA_HOME
          value: /usr/lib/jvm/java-1.8-openjdk/jre
        - name: profiler.micro.service.tag.trace.enable
          value: "true"
        image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-a:0.1-SNAPSHOT
        imagePullPolicy: Always
        name: spring-cloud-a-gray
        ports:
        - containerPort: 20001
          protocol: TCP
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
        livenessProbe:
          tcpSocket:
            port: 20001
          initialDelaySeconds: 10
          periodSeconds: 30
# Application B baseline version, with graceful shutdown disabled.
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: spring-cloud-b
  name: spring-cloud-b
spec:
  replicas: 2
  selector:
    matchLabels:
      app: spring-cloud-b
  strategy:
  template:
    metadata:
      labels:
        app: spring-cloud-b
        msePilotCreateAppName: spring-cloud-b
        msePilotAutoEnable: "on"
    spec:
      containers:
      - env:
        - name: LANG
          value: C.UTF-8
        - name: JAVA_HOME
          value: /usr/lib/jvm/java-1.8-openjdk/jre
        - name: micro.service.shutdown.server.enable
          value: "false"
        - name: profiler.micro.service.http.server.enable
          value: "false"
        image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-b:0.1-SNAPSHOT
        imagePullPolicy: Always
        name: spring-cloud-b
        ports:
        - containerPort: 8080
          protocol: TCP
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
        livenessProbe:
          tcpSocket:
            port: 20002
          initialDelaySeconds: 10
          periodSeconds: 30
# Application B canary version, with graceful shutdown enabled by default.
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: spring-cloud-b-gray
  name: spring-cloud-b-gray
spec:
  replicas: 2
  selector:
    matchLabels:
      app: spring-cloud-b-gray
  template:
    metadata:
      labels:
        alicloud.service.tag: gray
        app: spring-cloud-b-gray
        msePilotCreateAppName: spring-cloud-b
        msePilotAutoEnable: "on"
    spec:
      containers:
      - env:
        - name: LANG
          value: C.UTF-8
        - name: JAVA_HOME
          value: /usr/lib/jvm/java-1.8-openjdk/jre
        image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-b:0.1-SNAPSHOT
        imagePullPolicy: Always
        name: spring-cloud-b-gray
        ports:
        - containerPort: 8080
          protocol: TCP
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
        livenessProbe:
          tcpSocket:
            port: 20002
          initialDelaySeconds: 10
          periodSeconds: 30
# Application C baseline version
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: spring-cloud-c
  name: spring-cloud-c
spec:
  replicas: 2
  selector:
    matchLabels:
      app: spring-cloud-c
  template:
    metadata:
      labels:
        app: spring-cloud-c
        msePilotCreateAppName: spring-cloud-c
        msePilotAutoEnable: "on"
    spec:
      containers:
      - env:
        - name: LANG
          value: C.UTF-8
        - name: JAVA_HOME
          value: /usr/lib/jvm/java-1.8-openjdk/jre
        image: registry.cn-shanghai.aliyuncs.com/yizhan/spring-cloud-c:0.1-SNAPSHOT
        imagePullPolicy: Always
        name: spring-cloud-c
        ports:
        - containerPort: 8080
          protocol: TCP
        resources:
          requests:
            cpu: 250m
            memory: 512Mi
        livenessProbe:
          tcpSocket:
            port: 20003
          initialDelaySeconds: 10
          periodSeconds: 30
# HPA configuration
---
apiVersion: autoscaling.alibabacloud.com/v1beta1
kind: CronHorizontalPodAutoscaler
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: spring-cloud-b
spec:
   scaleTargetRef:
      apiVersion: apps/v1beta2
      kind: Deployment
      name: spring-cloud-b
   jobs:
   - name: "scale-down"
     schedule: "0 0/5 * * * *"
     targetSize: 1
   - name: "scale-up"
     schedule: "10 0/5 * * * *"
     targetSize: 2
---
apiVersion: autoscaling.alibabacloud.com/v1beta1
kind: CronHorizontalPodAutoscaler
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: spring-cloud-b-gray
spec:
   scaleTargetRef:
      apiVersion: apps/v1beta2
      kind: Deployment
      name: spring-cloud-b-gray
   jobs:
   - name: "scale-down"
     schedule: "0 0/5 * * * *"
     targetSize: 1
   - name: "scale-up"
     schedule: "10 0/5 * * * *"
     targetSize: 2
---
apiVersion: autoscaling.alibabacloud.com/v1beta1
kind: CronHorizontalPodAutoscaler
metadata:
  labels:
    controller-tools.k8s.io: "1.0"
  name: spring-cloud-c
spec:
   scaleTargetRef:
      apiVersion: apps/v1beta2
      kind: Deployment
      name: spring-cloud-c
   jobs:
   - name: "scale-down"
     schedule: "0 2/5 * * * *"
     targetSize: 1
   - name: "scale-up"
     schedule: "10 2/5 * * * *"
     targetSize: 2
# Expose Zuul gateway via SLB for demonstration
---
apiVersion: v1
kind: Service
metadata:
  name: zuul-slb
spec:
  ports:
    - port: 80
      protocol: TCP
      targetPort: 20000
  selector:
    app: spring-cloud-zuul
  type: ClusterIP
# Expose Application A as a Kubernetes Service
---
apiVersion: v1
kind: Service
metadata:
  name: spring-cloud-a-base
spec:
  ports:
    - name: http
      port: 20001
      protocol: TCP
      targetPort: 20001
  selector:
    app: spring-cloud-a
---
apiVersion: v1
kind: Service
metadata:
  name: spring-cloud-a-gray
spec:
  ports:
    - name: http
      port: 20001
      protocol: TCP
      targetPort: 20001
  selector:
    app: spring-cloud-a-gray
# Nacos Server SLB Service configuration
---
apiVersion: v1
kind: Service
metadata:
  name: nacos-slb
spec:
  ports:
  - port: 8848
    protocol: TCP
    targetPort: 8848
  selector:
    app: nacos-server
  type: LoadBalancer

Enable microservice governance for your deployed applications. For details, see Enable MSE governance for ACK and ACS microservice applications (Java).

View observable data

Both spring-cloud-b and spring-cloud-b-gray have CronHPA enabled, simulating scheduled scaling every 5 minutes. Click an application name, then click the Scaling tab to view details.

spring-cloud-b application:

On the CronHPA page, spring-cloud-b has two scheduled scaling tasks, both successfully executed (status: Succeed). Task scale-down runs on schedule 0 0/5 * * * * with a target replica count of 1. Task scale-up runs on schedule 10 0/5 * * * * with a target replica count of 2.
spring-cloud-b-gray application:

The CronHPA page shows two scheduled tasks: scale-down (schedule 0 0/5 * * * *, target replica count 1) and scale-up (schedule 10 0/5 * * * *, target replica count 2), both with status Succeed.

Log on to the MSE console, and select a region in the top navigation bar.
In the navigation pane on the left, choose Microservices Governance > Application Governance, then click the spring-cloud-a application card.
The Application Overview page displays observable data for this application.
The data shows:
- The spring-cloud-a-gray version had zero request errors during pod scaling, indicating no traffic loss.
- The spring-cloud-a version, which has graceful shutdown disabled, experienced 20 failed requests from spring-cloud-a to spring-cloud-b during pod scaling, resulting in traffic loss.

Enable graceful start

The spring-cloud-c application uses CronHPA to simulate startup behavior, scaling every 5 minutes—scaling down to one node at second 0 of minute 2, and scaling up to two nodes at second 10 of minute 2. On the CronHPA page, the spring-cloud-c CronHPA has two tasks: scale-down (schedule 0 2/5 * * * *, target replica count 1, status Succeed) and scale-up (schedule 10 2/5 * * * *, target replica count 2, status Succeed).

Log on to the MSE console, and select a region in the top navigation bar.
In the navigation pane on the left, choose Microservices Governance > Application Governance, then click the spring-cloud-c application card.
In the navigation pane on the left, choose Traffic management, then select the Graceful Start/Shutdown tab. Turn on Graceful Start. In the Prompt message dialog box, click OK. The warmup duration defaults to 120 seconds.

After enabling warmup, traffic to a restarted application gradually increases over time. In slow-start scenarios—such as when an application needs to pre-build connection pools or caches during startup—enabling service warmup ensures orderly resource initialization and safe, traffic-loss-free startup.