Elastic Container Instance:Set a custom maximum Pending duration

Overview

Each ECI instance corresponds to a pod. When you create an ECI pod, billing starts after the corresponding ECI instance transitions from the Scheduling state to the Pending state. During the Pending phase, if issues such as image pull failures or volume mount failures occur, the ECI instance remains in the Pending state and continues to incur charges. By default, the system automatically terminates an ECI instance and stops billing if its Pending duration exceeds 4 hours. If you have specific time requirements, you can customize the maximum Pending duration.

Configuration

You can add thek8s.aliyun.com/eci-max-pending-minute annotation to the pod metadata to customize the maximum Pending duration for the corresponding ECI instance.

• The value must be an integer from 10 to 1,440 minutes (1 day).

  Important

  Set a reasonable maximum Pending duration based on your business requirements. If an image is large and no image cache is available, pulling the image can take a significant amount of time. If the maximum Pending duration is too short, the pod might be terminated before the image pull completes.

• If this annotation is not added, the default maximum Pending duration is 4 hours.

• This annotation does not apply to an ECI pod that has a running init container.

After the maximum Pending duration is exceeded, the system reports a corresponding event. The ECI instance status changes to Failed, and the pod status (PodStatus.Phase) is determined by the restartPolicy.

• If restartPolicy is set to Always or OnFailure, PodStatus.Phase is Pending.

• If restartPolicy is set to Never, PodStatus.Phase is Failed.

Configuration example

1. Create an ECI pod with a specified maximum Pending duration.

   kubectl create -f pending-test.yaml

   The following sample pending-test.yaml file simulates a pod stuck in the Pending state due to an image pull failure.

   apiVersion: v1
   kind: Pod
   metadata:
     name: pending-test
     labels:
       alibabacloud.com/eci: "true"
     annotations:
       k8s.aliyun.com/eci-max-pending-minute: "10"    # Set the maximum Pending duration to 10 minutes.
   spec:
     containers:
     - image: test****-registry.example.com/eci_test/nginx:1.0  # Use a private image to simulate an image pull failure. Replace the image with your actual image.
       name: test-container
     restartPolicy: Never  # Set the restart policy to Never.

2. Check the ECI pod status.

   kubectl get pod <pod-name> -o=jsonpath='{.status.phase}'

   Sample output:

   :~$ kubectl get pod
   NAME            READY   STATUS             RESTARTS   AGE
   pending-test    0/1     ImagePullBackOff   0          25s
   :~$ kubectl get pod pending-test -o=jsonpath='{.status.phase}'
   Pending

3. Wait for the specified maximum Pending duration to elapse, and then check the status and events of the ECI pod.

   • Because restartPolicy is set to Never, the pod enters the Failed state after the timeout.

     kubectl get pod <pod-name> -o=jsonpath='{.status.phase}'

     Sample output:

     :~$ kubectl get pod
     NAME             READY   STATUS                              RESTARTS   AGE
     pending-test     0/1     ContainerInstanceStartingFailed     0          15m
     :~$ kubectl get pod pending-test -o=jsonpath='{.status.phase}'
     Failed

   • Check the pod events.

     kubectl get events --field-selector involvedObject.name=<pod-name>

     Sample output:

     Warning   Failed           pod/pending-test   Error: ErrImagePull
     Normal    BackOff          pod/pending-test   Back-off pulling image "test xxx om/eci_test/nginx:1.0"
     Warning   Failed           pod/pending-test   Error: ImagePullBackOff
     Warning   StartingFailed   pod/pending-test   [eci.containergroup]eci starting failed: The Eci pod stuck in Pending exceed eci-max-pending-minute 10, will be released.

   Note

   You can also use the Elastic Container Instance console to check if the ECI instance is in the Failed state. On the instance details page, open the Events tab to view the events.