All Products
Search
Document Center

Use probes to perform health checks on containers

Last Updated: May 18, 2021

This topic describes how to configure liveness probes and readiness probes to perform health checks on containers.

Background information

In Kubernetes, the kubelet uses liveness and readiness probes to periodically check the status and running conditions of containers.

  • Liveness probe

    Liveness probes are used to check whether a container is working normally. If the check succeeds, the container is working normally. If the check fails, the system determines whether to restart the container based on the configured container restart policy. By default, if a liveness probe is not configured, the container is considered to be working normally all the time.

    Liveness probes can be applied in the following scenarios:

    • When an application is running but no further operations can be performed on it, liveness probes can capture the deadlock. The system then restarts the container to make the application run normally despite bugs.

    • Many applications running for a long period of time may eventually transition to the broken state, and cannot recover except by being restarted. You can use liveness probes to detect and remedy such situations.

  • Readiness probe

    Readiness probes are used to check whether a container is ready to serve requests. If the check succeeds, the container is ready to receive business requests. If the check fails, the container is not ready and the system stops sending requests to the container until the recheck succeeds.

    Readiness probes can be applied in the following scenarios:

    Applications are temporarily unable to serve external traffic. For example, an application may need to load a large amount of data or configuration files during startup. In this case, if you do not want to terminate the application or send it requests, you can use readiness probes to detect and mitigate such situations.

Kubernetes mode

When you create an elastic container instance by using Kubernetes, you can configure liveness probes and readiness probes by using the livenessProbe and readinessProbe fields of the container. Examples:

  • Configure a liveness probe

    apiVersion: v1
    kind: Pod
    metadata:
      labels:
        test: liveness
      name: liveness-exec
    spec:
      containers:
      - name: liveness
        image: busybox:latest
        args:
        - /bin/sh
        - -c
        - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
        # Configure a liveness probe to check containers by using the command line.
        livenessProbe:    
          exec:           
            command:
            - cat
            - /tmp/healthy
          initialDelaySeconds: 5    #Start to check a container 5 seconds after the container is started.
          periodSeconds: 5    #Perform a check every 5 seconds.
  • Configure a readiness probe

    apiVersion: v1
    kind: Pod
    metadata:
      labels:
        test: readiness
      name: readiness-exec
    spec:
      containers:
      - name: readiness
        image: busybox:latest
        args:
        - /bin/sh
        - -c
        - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
        # Configure a readiness probe to check containers by using the command line.
        readinessProbe:
          exec:           
            command:
            - cat
            - /tmp/healthy
          initialDelaySeconds: 5    #Start to check a container 5 seconds after the container is started.
          periodSeconds: 5    #Perform a check every 5 seconds.

For more information, see Configure Liveness, Readiness and Startup Probes.

API mode

When you create an elastic container instance by calling the CreateContainerGroup operation, you can use the LivenessProbe and ReadinessProbe parameters to configure liveness probes and readiness probes. The following table describes the parameters. For more information, see CreateContainerGroup.

  • LivenessProbe related parameters

Parameter

Type

Required

Example

Description

Container.N.LivenessProbe.HttpGet.Path

String

No

/healthyz

The path to which HTTP GET requests are sent for health checks.

Container.N.LivenessProbe.HttpGet.Port

Integer

No

8888

The port to which HTTP GET requests are sent for health checks.

Container.N.LivenessProbe.HttpGet.Scheme

String

No

HTTP

The protocol type of HTTP GET requests when you use HTTP requests for health checks. Valid values:

  • HTTP

  • HTTPS

Container.N.LivenessProbe.InitialDelaySeconds

Integer

No

5

The number of seconds after the container is started before a check is started.

Container.N.LivenessProbe.PeriodSeconds

Integer

No

1

The interval for checking the container. Default value: 10. Minimum value: 1. Unit: seconds.

Container.N.LivenessProbe.SuccessThreshold

Integer

No

1

The minimum number of consecutive times that a health check must succeed after a failed check before the check is declared successful. Default value: 1. Set the value to 1.

Container.N.LivenessProbe.FailureThreshold

Integer

No

3

The minimum number of consecutive times that a health check must fail after a successful check before the check is declared failed. Default value: 3.

Container.N.LivenessProbe.TimeoutSeconds

Integer

No

1

The timeout period of the check. Default value: 1. Minimum value: 1. Unit: seconds.

Container.N.LivenessProbe.Exec.Command.N

RepeatList

No

cat /tmp/healthy

Command N to be executed in the container when the health check is performed by using the command line.

Container.N.LivenessProbe.TcpSocket.Port

Integer

No

8000

The port detected by Transmission Control Protocol (TCP) socket when health checks are performed by using TCP socket.

  • ReadinessProbe related parameters

Parameter

Type

Required

Example

Description

Container.N.ReadinessProbe.HttpGet.Path

String

No

/healthyz

The path to which HTTP GET requests are sent for health checks.

Container.N.ReadinessProbe.HttpGet.Port

Integer

No

8888

The port to which HTTP GET requests are sent for health checks.

Container.N.ReadinessProbe.HttpGet.Scheme

String

No

HTTP

The protocol type of HTTP GET requests when you use HTTP requests for health checks. Valid values:

  • HTTP

  • HTTPS

Container.N.ReadinessProbe.InitialDelaySeconds

Integer

No

5

The number of seconds after the container is started before probes are started.

Container.N.ReadinessProbe.PeriodSeconds

Integer

No

1

The interval for checking the container. Default value: 10. Minimum value: 1. Unit: seconds.

Container.N.ReadinessProbe.SuccessThreshold

Integer

No

1

The minimum number of consecutive times that a health check must succeed after a failed check before the check is declared successful. Default value: 1. Set the value to 1.

Container.N.ReadinessProbe.FailureThreshold

Integer

No

3

The minimum number of consecutive times that a health check must fail after a successful check before the check is declared failed. Default value: 3.

Container.N.ReadinessProbe.TimeoutSeconds

Integer

No

1

The timeout period of the check. Default value: 1. Minimum value: 1. Unit: seconds.

Container.N.ReadinessProbe.Exec.Command.N

RepeatList

No

cat /tmp/healthy

Command N to be executed in the container when the health check is performed by using the command line.

Container.N.ReadinessProbe.TcpSocket.Port

Integer

No

8000

The port detected by TCP socket when health checks are performed by using TCP socket.

Console mode

When you create an elastic container instance in the Elastic Container Instance console, you can click Advanced Settings in the Container Configurations section and enable Health Check. The following figure shows the configuration details.

Note

When you configure health check in the Elastic Container Instance console, only the command line and HTTP request methods are supported. The TCP socket method is not supported.

Health check 1

The following table describes the parameters.

Parameter

Description

Time Settings

You must set Waiting Period and Timeout Period.

  • Waiting Period: The number of seconds to wait before the probe performs the first health check.

  • Timeout Period: The timeout period of the check. If a timeout error occurs, the check failed.

Method

Valid values:

  • Script: The probe runs commands in the container and checks the exit code of the command. If the exit code is 0, the check is successful.

  • HTTP Request Method: The probe sends an HTTP request to the container. If the returned status code is greater than or equal to 200 and less than 400, the check is successful.

Script

When you select Script for Method, you must configure the command line script to be executed in the container.

HTTP Request Method

When you select HTTP Request Method for Method, you must configure the path, port, and protocol for HTTP GET requests.

Configuration examples

In this example, an elastic container instance that has a liveness probe and readiness probe configured is created and contains an NGINX container. Then, service exceptions are simulated to check whether the configured probes take effect.

  1. Create an elastic container instance by using SDK for Java.

    Create an elastic container instance by using an NGINX image. The following sample code shows how to configure a liveness probe and readiness probe:

    // Configure a liveness probe. After the container is running for 5 seconds, the kubelet runs the liveness probe on port 80 every 3 seconds. The timeout period of each probe is set to 10 seconds. The number of consecutive successes for a probe to be considered successful is set to 3, and the number of consecutive failures for a probe to considered failed is also set to 3.
    CreateContainerGroupRequest.Container.ContainerProbe livenessProbe = new CreateContainerGroupRequest.Container.ContainerProbe();
    livenessProbe.setTcpSocketPort(80);
    livenessProbe.setInitialDelaySeconds(5);
    livenessProbe.setPeriodSeconds(3);
    livenessProbe.setFailureThreshold(3);
    livenessProbe.setSuccessThreshold(1);
    livenessProbe.setTimeoutSeconds(10);
    // Configure a readiness probe. After the container is running for 5 seconds, the kubelet runs the readiness probe on port 80 every 3 seconds. The timeout period of each check is set to 10 seconds. The number of consecutive times that a health check must succeed before the check declared successful is set to 1, and the number of consecutive times that a probe must fail before the check is declared failed is set to 3.
    CreateContainerGroupRequest.Container.ContainerProbe readinessProbe = new CreateContainerGroupRequest.Container.ContainerProbe();
    readinessProbe.setTcpSocketPort(80);
    readinessProbe.setInitialDelaySeconds(5);
    readinessProbe.setPeriodSeconds(3);
    readinessProbe.setFailureThreshold(3);
    readinessProbe.setSuccessThreshold(3);
    readinessProbe.setTimeoutSeconds(10);
  2. View the related events after an elastic container instance is created.

    After an elastic container instance is created, view events about the instance and you can see that the instance starts normally.

    Health check 2
  3. Change the NGINX listening port in the configuration file to simulate a service exception.

    1. Change the NGINX listening port.

      vi /etc/nginx/conf.d/default.conf

      Example:

      Health check 3
    2. Restart NGINX.

      nginx -s reload
  4. View the effective status of the probe.

    A few seconds after you restart NGINX, the container automatically restarts. View events about the instance, and you can see that the container is restarted after the liveness probe and readiness probe each have three consecutive failures.

    Health check 4