When you create elastic container instance-based pods in a Kubernetes cluster, you can add annotations to the pods to use the features of Elastic Container Instance. Make sure that the annotations that you want to add comply with the Kubernetes syntax. This topic describes the annotations that are supported by elastic container instance-based pods and the annotations that you can add after the system schedules resources and creates pods. This topic also provides examples on how to configure the annotations.
Annotations that you can add when you create pods
The following table describes the annotations that you can add when you create pods.
The annotations described in the following table are applicable only to the pods that are scheduled to VNodes. These pods run as elastic container instances. The annotations cannot be added to the pods that are scheduled to real nodes.
Annotations must be added to the metadata of the configuration file of the pods. For example, when you create a Deployment, you must add annotations in the spec.template.metadata section.
To use features of Elastic Container Instance, you can add annotations only when you create Elastic Container Instance-based pods. If you add or modify annotations when you update pods, these annotations do not take effect.
Annotation | Example | Description | References |
k8s.aliyun.com/eci-security-group | sg-bp1dktddjsg5nktv**** | Specifies the ID of the security group. The following requirements must be met:
| |
k8s.aliyun.com/eci-vswitch | vsw-bp1xpiowfm5vo8o3c**** | Specifies vSwitch IDs. You can specify multiple vSwitch IDs to ensure that pods can be created in zones in which sufficient resources exist. | |
k8s.aliyun.com/eci-schedule-strategy | VSwitchOrdered | Configures a multi-zone scheduling policy. Valid values:
| |
k8s.aliyun.com/eci-use-specs | 2-4Gi,4-8Gi,ecs.c6.xlarge | Specifies one or more specifications of the pod. The specifications can be a combination of number of vCPUs and memory size. The specifications can also be instance types of Elastic Compute Service (ECS) instances. | |
k8s.aliyun.com/eci-gpu-driver-version | tesla=525.85.12 | Specifies the version of the GPU driver. If you specify a GPU-accelerated ECS instance family that supports multiple versions of GPU drivers and CUDA when you create a GPU-accelerated elastic container instance, you can configure this annotation to specify the version of the driver and CUDA. | |
k8s.aliyun.com/eci-instance-family | "ecs.c6,ecs.g6" | Specifies or excludes specific ECS instance families while you create pods by specifying specifications of vCPU and memory. | |
k8s.aliyun.com/eci-instance-generation | "6,5" | Specifies or excludes specific generations of ECS instance families while you create pods by specifying specifications of vCPU and memory. | Specify or exclude specific generations of ECS instance families to create a pod |
k8s.aliyun.com/eci-spot-strategy | SpotAsPriceGo | Specifies the bid policy for the preemptible instance. Valid values:
| |
k8s.aliyun.com/eci-spot-price-limit | "0.5" | Specifies the maximum hourly price of the preemptible instance. This value can be accurate to up to three decimal places. This annotation is valid only when k8s.aliyun.com/eci-spot-strategy is set to SpotWithPriceLimit. | |
k8s.aliyun.com/eci-spot-duration | "0" | Specifies the protection period of the preemptible instance. Unit: hour. Default value: 1. A value of 0 indicates no protection period. | |
k8s.aliyun.com/eci-spot-fallback | "true" | Specifies whether to automatically create a pay-as-you-go instance if inventory resources that meet the requirements for the preemptible instance specification are insufficient. Default value: false. | |
k8s.aliyun.com/eci-privatepool-matchcriteria | open | Specifies the matching mode in which the system matches private pools. Valid values:
| Use the private pool of an elasticity assurance to create pods |
k8s.aliyun.com/eci-privatepool-id | eap-2ze1g68k2melxkkl**** | Specifies a private pool ID (the elasticity assurance ID). You can obtain the private pool ID on the Resource Reservations or Private Pools tab of the Resource Reservations page in the ECS console.
| |
k8s.aliyun.com/eci-custom-tags | "env:test,name:alice" | Specifies the tag string. You can bind a maximum of three tags. Separate a tag key and a tag value with a colon (:). Separate multiple tags with commas (,). | |
k8s.aliyun.com/eci-ram-role-name | AliyunECIContainerGroupRole | Binds a RAM role that the pod can assume to access other Alibaba Cloud services. | |
k8s.aliyun.com/eci-fail-strategy | fail-back | Configures a fault handling policy for the pod. Valid values:
| |
k8s.aliyun.com/eci-custom-hosts | "[{\"host\":\"example.com\",\"ip\":\"100.100.XX.XX\"},{\"host\":\"aliyundoc.com\",\"ip\":\"100.100.XX.XX\"}]" | Configures hosts (/etc/hosts) for pods. | |
k8s.aliyun.com/eci-auto-imc | "true" | Specifies whether to automatically match image caches. | |
k8s.aliyun.com/imc-perfect-match | "true" | Specifies whether all container images in the pod must exactly match the image cache. | |
k8s.aliyun.com/imc-match-count-request | "2" | Specifies the number of container images in the pod that you want to exactly match the image cache. | |
k8s.aliyun.com/eci-imc-id | imc-2zebxkiifuyzzlhl**** | Specifies the ID of the image cache. | |
k8s.aliyun.com/acr-instance-ids | cri-j36zhodptmyq**** | Specifies the IDs of Alibaba Cloud Container Registry Enterprise Edition instances. These instances are used to pull images without using Secrets. You can specify Container Registry Enterprise Edition instances that reside in a region different from the region of the pod. You must prefix the region ID of an Enterprise Edition instance to the ID of the Enterprise Edition instance. Example: | Pull images from a Container Registry instance without using a Secret |
k8s.aliyun.com/acr-service-arns | acs:ram::1609982529******:role/role-assume | Specifies the Alibaba Cloud Resource Names (ARNs) of the RAM roles in the Alibaba Cloud account to which the Elastic Container Instance resources belong. This annotation is required when you use a RAM role to create resources. | |
k8s.aliyun.com/acr-user-arns | acs:ram::1298452580******:role/role-acr | Specifies the ARNs of the RAM roles in the Alibaba Cloud account to which the Container Registry instance belongs. This annotation is required when you pull images from a Container Registry instance that belongs to an Alibaba Cloud account different from the Alibaba Cloud account of the Elastic Container Instance resource. | |
k8s.aliyun.com/eci-data-cache-bucket | default | Specifies the bucket that is used to store the DataCache. You must configure this annotation when you use DataCaches to create pods. | |
k8s.aliyun.com/eci-data-cache-pl | PL1 | Specifies the performance level of the disk that is created based on the DataCache. By default, a PL1 enhanced SSD (ESSD) is used. | |
k8s.aliyun.com/eci-data-cache-provisionedIops | "40000" | Specifies the read/write IOPS that is provisioned for the ESSD AutoPL disk. Valid values: 0 to min{50000, 1000 × Storage capacity - Baseline IOPS}. Baseline IOPS = min{1,800 + 50 × Storage capacity, 50,000}. For more information, see ESSD AutoPL disks. If you add this annotation, the disk that is created based on the DataCache is an ESSD AutoPL disk. | |
k8s.aliyun.com/eci-data-cache-burstingEnabled | "true" | Specifies whether to enable the performance burst feature for the ESSD AutoPL disk. For more information, see ESSD AutoPL disks. If you add this annotation, the disk that is created based on the DataCache is an ESSD AutoPL disk. | |
k8s.aliyun.com/eci-eip-instanceid | eip-bp1q5n8cq4p7f6dzu**** | Specifies the ID of the elastic IP address (EIP). | |
k8s.aliyun.com/eci-with-eip | "true" | Specifies whether to automatically create an EIP when you create an elastic container instance and associate the EIP with the instance. | |
k8s.aliyun.com/eip-bandwidth | "5" | Specifies the bandwidth limit for the EIP. Unit: Mbit/s. Default value: 5. | |
k8s.aliyun.com/eip-common-bandwidth-package-id | cbwp-2zeukbj916scmj51m**** | Specifies the ID of an existing EIP bandwidth plan that you want to associate with the instance. For more information, see What is an Internet Shared Bandwidth? | |
k8s.aliyun.com/eip-isp | BGP | Specifies the line type for the EIP. This annotation is applicable only to pay-as-you-go EIPs. Valid values:
For more information, see the "Line types" section of the Elastic IP Addresses topic. | |
k8s.aliyun.com/eip-internet-charge-type | PayByBandwidth | Specifies the metering method of the EIP. Valid values:
For more information about the billing of EIPs, see Billing overview. | |
k8s.aliyun.com/eip-public-ip-address-pool-id | pippool-bp187arfugi543y1s**** | Specifies the ID of the IP address pool. The EIP is allocated from the IP address pool. For more information, see Create and manage IP address pools. | |
k8s.aliyun.com/eci-enable-ipv6 | "true" | Specifies whether to assign an IPv6 address to the pod. | Assign an IPv6 address to an Elastic Container Instance-based pod |
k8s.aliyun.com/eci-ipv6-bandwidth-enable | "true" | Specifies whether to enable Internet access over IPv6 addresses for the instance. | |
k8s.aliyun.com/eci-ipv6-bandwidth | 100M | Specifies the maximum public bandwidth of the IPv6 address. | |
k8s.aliyun.com/eci-private-ip-address | "172.16.0.1" | Specifies the private IP address of the pod. Only IPv4 addresses are supported. Make sure that the IP address is idle. | |
k8s.aliyun.com/eci-fixed-ip | "true" | Specifies whether to configure the pod to use a fixed IP address. | Configure an Elastic Container Instance-based pod to use a fixed IP address |
k8s.aliyun.com/eci-fixed-ip-retain-hour | "24" | Specifies the retention period of the fixed IP address after the pod with using the fixed IP address enabled is released and the fixed IP address becomes idle. Unit: hours. If you do not configure the annotation, the fixed IP address is retained for 48 hours by default. | |
kubernetes.io/ingress-bandwidth | 40M | Specifies the inbound bandwidth. | |
kubernetes.io/egress-bandwidth | 20M | Specifies the outbound bandwidth. | |
k8s.aliyun.com/eci-extra-ephemeral-storage | 50Gi | Specifies the size of the temporary storage space. | |
k8s.aliyun.com/eci-ephemeral-storage-options | "{\"encrypted\":\"true\"}" | Specifies the parameters for the temporary storage space. A value of | |
k8s.aliyun.com/eci-eviction-enable | "true" | Specifies whether to automatically evict the pods that have insufficient temporary storage space. | Automatically evict pods whose temporary storage spaces are insufficient |
k8s.aliyun.com/eci-core-pattern | /pod/data/dump/core | Specifies the directory in which core dump files are stored. | |
k8s.aliyun.com/eci-ntp-server | 100.100.*.* | Specifies the IP address of the Network Time Protocol (NTP) server. | |
k8s.aliyun.com/plain-http-registry | harbor***.pre.com,192.168.XX.XX:5000,reg***.test.com:80 | Configures the address of the self-managed image repository. When you create a pod by using an image in a self-managed image repository that uses the HTTP protocol, you must add this annotation to the instance. This way, Elastic Container Instance pulls the image over the HTTP protocol instead of the default HTTPS protocol. This prevents image pull failures caused by different protocols. | |
k8s.aliyun.com/insecure-registry | harbor***.pre.com,192.168.XX.XX:5000,reg***.test.com:80 | Configures the address of the self-managed image repository. When you create a pod by using an image in a self-managed image repository that uses a self-signed certificate, you must add this annotation to the instance to skip the certificate authentication. This prevents image pull failures caused by certificate authentication failures. | |
k8s.aliyun.com/eci-sls-enable | "false" | Specifies whether to enable the log collection feature. Specifies whether to collect logs for the pod. If you do not want to collect logs for the pod when you use Simple Log Service CRDs (custom resource definitions) to collect logs, you can add the annotation to the pod to disable the log collection feature. This prevents waste of resources when the system automatically creates Logtail. |
Annotations that you can add after the system schedules resources and creates pods
The following table describes the annotations that you can add after the system schedules resources and creates pods. You can run the kubectl describe
command to query the details of the pods.
Annotation | Example | Description |
k8s.aliyun.com/eci-request-id | 45942504-4688-51BA-BBAB-4B692C4F39C0 | Indicates the request ID. |
k8s.aliyun.com/eci-instance-id | eci-2ze1y0la40qgva09**** | Indicates the pod ID. |
k8s.aliyun.com/eci-instance-spec | 2.0-4.0Gi | Indicates the specifications of the pod. You are charged based on the value.
|
k8s.aliyun.com/allocated-eipInstanceId | eip-bp1q5n8cq4p7f6dzu**** | Indicates the EIP ID. |
k8s.aliyun.com/allocated-eipAddress | 47.99.**.** | Indicates the EIP that is associated with the instance. |
k8s.aliyun.com/allocated-ipv6Address | 2001:d**:1:1:1:1:1:1 | Indicates the IPv6 address that is assigned to the pod. |
k8s.aliyun.com/eci-created-by-template | true | Indicates whether a template was used to create the pod. |
k8s.aliyun.com/eni-instance-id | eni-2ze6d7oo5ukqj26o**** | Indicates the ID of the elastic network interface (ENI) that is bound to the pod. |
k8s.aliyun.com/eci-vpc | vpc-2zeghwzptn5zii0w7**** | Indicates the virtual private cloud (VPC) to which the pod belongs. |
k8s.aliyun.com/eci-matched-image-cache | imc-2zedy3v37800iimu**** | Indicates the ID of the matched image cache. |
k8s.aliyun.com/eci-schedule-result | finished | Indicates the scheduling result. |
Configure rescheduling for elastic container instances
Pods may fail to be scheduled to virtual nodes. You can add an annotation to enable rescheduling for pods. This ensures that the system keeps scheduling pods instead of returning failures even if the asynchronous scheduling fails.
Example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: set-eci
labels:
app: vk
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
annotations:
k8s.aliyun.com/eci-reschedule-enable: "true" # Enable rescheduling for elastic container instances.
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
nodeName: virtual-kubelet
Set the pods that cannot be created to the Failed state
By default, if an error occurs when a pod is being created, the system tries to recreate the pod for a specified number of times. If the error persists, the pod changes to the Pending state. However, if some job-type pods fail to be created, you may want to directly set the pods to the Failed state. To do this, you can add an annotation to set a pod that fails to be created to the Failed state.
Example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: set-pod-fail-on-create-err
labels:
app: vk
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
annotations:
k8s.aliyun.com/pod-fail-on-create-err: "true" # Set the state to Failed if the pod fails to be created.
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:latest
nodeName: virtual-kubelet