All Products
Search
Document Center

ECI Pod Annotation

Last Updated: Dec 01, 2021

When you use a virtual node to schedule pods of a Kubernetes cluster to Elastic Container Instance, you can add annotations to the pods. This way, you can make full use of the Elastic Container Instance features. Make sure that the annotations that you want to add comply with the Kubernetes syntax. This topic describes the annotations supported by Elastic Container Instance and provides examples on how to configure the annotations.

The following table describes the annotations supported by Elastic Container Instance.

Note

The annotations described in the following table are suitable only for the pods that are scheduled to Elastic Container Instance by using virtual nodes. These pods are run on elastic container instances. The annotations cannot be added to the pods that are scheduled to Elastic Compute Service (ECS).

Annotation

Example

Description

References

k8s.aliyun.com/eci-security-group

sg-bp1dktddjsg5nktv****

The ID of the security group.

Configure a security group

k8s.aliyun.com/eci-vswitch

vsw-bp1xpiowfm5vo8o3c****

The IDs of vSwitches. You can specify multiple vSwitches for multiple zones.

Specify multiple zones to create an elastic container instance

k8s.aliyun.com/eci-schedule-strategy

VSwitchOrdered

The multi-zone scheduling policy. Default value: NoSpot. Valid values:

  • VSwitchOrdered: Resources in the specified zones are scheduled in the order in which the vSwitches are specified.

  • VSwitchRandom: Resources in the specified zones are scheduled in a random manner.

k8s.aliyun.com/eci-ram-role-name

AliyunECIContainerGroupRole

The Resource Access Management (RAM) role that grants Elastic Container Instance permissions to access other Alibaba Cloud services.

None. The following section describes the details.

k8s.aliyun.com/eci-use-specs

2-4Gi,4-8Gi,ecs.c6.xlarge

The elastic container instance types. You can specify multiple elastic container instance types. An elastic container instance type can be a combination of vCPUs and memory or an ECS instance type.

Specify multiple instance types to create an elastic container instance

k8s.aliyun.com/eci-spot-strategy

SpotAsPriceGo

The bidding policy for the preemptible instance. Default value: NoSpot. Valid values:

  • SpotAsPriceGo: The system places bids based on the spot price.

  • SpotWithPriceLimit: You must specify the maximum price that you want to pay for the preemptible instance per hour.

Create a preemptible instance

k8s.aliyun.com/eci-spot-price-limit

0.5

The maximum price of the preemptible instance per hour. This parameter is valid only when k8s.aliyun.com/eci-spot-strategy is set to SpotWithPriceLimit.

k8s.aliyun.com/eci-cpu-option-core

2

The number of physical CPU cores.

Customize CPU options

k8s.aliyun.com/eci-cpu-option-ht

1

The number of threads per core.

k8s.aliyun.com/eci-reschedule-enable

"true"

Specifies whether to enable rescheduling for elastic container instances.

None. See the sections that follow.

k8s.aliyun.com/pod-fail-on-create-err

"true"

Specifies whether to put the elastic container instances that cannot be created into the Failed state.

None. The following section describes the details.

k8s.aliyun.com/eci-image-snapshot-id

imc-2zebxkiifuyzzlhl****

The ID of the image cache.

Note

You can specify image caches or enable automatic matching for image caches. We recommend that you enable automatic matching for image caches.

Use an image cache CRD to accelerate pod creation

k8s.aliyun.com/eci-image-cache

"true"

Specifies whether to enable automatic matching for image caches.

Note

You can specify image caches or enable automatic matching for image caches. We recommend that you enable automatic matching for image caches.

k8s.aliyun.com/acr-instance-id

cri-j36zhodptmyq****

The ID of the Container Registry Enterprise Edition instance.

Configure settings without the need to use a password to pull images from Container Registry Enterprise Edition instances

k8s.aliyun.com/eci-eip-instanceid

eip-bp1q5n8cq4p7f6dzu****

The ID of the elastic IP address (EIP).

Enable Internet access

k8s.aliyun.com/eci-with-eip

"true"

Specifies whether to automatically create and associate an EIP.

k8s.aliyun.com/eip-bandwidth

5

The bandwidth value for the EIP.

k8s.aliyun.com/eip-common-bandwidth-package-id

cbwp-2zeukbj916scmj51m****

The ID of the EIP bandwidth plan.

k8s.aliyun.com/eip-isp

BGP

The line type for the EIP. This parameter is suitable only for pay-as-you-go EIPs. Valid values:

  • BGP: BGP (Multi-ISP) line

  • BGP_PRO: BGP (Multi-ISP) Pro line

k8s.aliyun.com/eip-internet-charge-type

PayByBandwidth

The metering method of the EIP. Valid values:

  • PayByBandwidth

  • PayByTraffic

k8s.aliyun.com/eci-enable-ipv6

"true"

Specifies whether to allocate IPv6 addresses.

Note

Each pod can be allocated a single IPv6 address. k8s.aliyun.com/eci-enable-ipv6 and k8s.aliyun.can be used in the same manner. Specify one of these parameters to allocate an IPv6 address.

Assign an IPv6 address to an elastic container instance

k8s.aliyun.com/eci-ipv6-count

1

The number of IPv6 addresses. Set the value to 1.

Note

Each pod can be allocated a single IPv6 address. k8s.aliyun.com/eci-enable-ipv6 and k8s.aliyun.com/eci-ipv6-count can be used in the same manner. Specify one of these parameters to allocate an IPv6 address.

kubernetes.io/ingress-bandwidth

40M

The inbound bandwidth.

Limit the bandwidth of an elastic container instance

kubernetes.io/egress-bandwidth

20M

The outbound bandwidth.

k8s.aliyun.com/eci-extra-ephemeral-storage

50Gi

The size of the temporary storage space.

Create a custom temporary storage space

k8s.aliyun.com/eci-core-pattern

/pod/data/dump/core

The directory in which core dump files are stored.

View core dump files

k8s.aliyun.com/eci-ntp-server

100.100.*.*

The IP address of the Network Time Protocol (NTP) server.

Configure an NTP server for pods

Configure security groups

When Virtual Kubelet starts, it uses environment variables to configure a default security group. By default, all pods that are created on each virtual node use the security group that is configured by Virtual Kubelet. You can add annotations to specify a security group for a pod based on your business requirements.

Configuration example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  labels:
    app: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
        annotations: 
            k8s.aliyun.com/eci-security-group: "sg-bp1dktddjsg5nktv****" # Configure the security group. 
        labels:
            app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
      nodeName: virtual-kubelet

Create pods in multiple zones

You can specify multiple zones where you can create pods. Before you specify multiple zones, specify multiple vSwitches. The system then selects a zone based on resource availability to create the pod. For more information, see Specify multiple zones to create an elastic container instance.

Configuration example:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.aliyun.com/eci-vswitch: "vsw-bp1xpiowfm5vo8o3c****,vsw-bp1rkyjgr1xwoho6k****" # Specify multiple vSwitch IDs. 
    k8s.aliyun.com/eci-schedule-strategy: "VSwitchOrdered" # Configure the multi-zone scheduling policy. 
  name: nginx-test
spec:
  containers:
  - name: nginx
    image: nginx:latest

Configure a RAM role

You can add an annotation to configure a RAM role for a pod. The role grants the pod permissions to access Alibaba Cloud services.

Notice

Make sure that the configured RAM role can be used by ECS.

Configuration example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: set-ram-role
  labels:
    app: vk
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
        annotations: 
            k8s.aliyun.com/eci-ram-role-name : "AliyunECIContainerGroupRole"   #Specify a RAM role. 
        labels:
            app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
      nodeName: virtual-kubelet

Specify one or more instance types to create a pod

You can add an annotation to specify instance types that can be used to create a pod. If resources of a specified instance type are insufficient, the system scans other specified instance types to find an instance type that provides sufficient resources to create the pod. For more information, see Specify multiple instance types to create an elastic container instance.

Note

An elastic container instance type can be a combination of vCPUs and memory or an ECS instance type. You can specify special ECS instance types such as GPU-accelerated ECS instances types, ECS instance types that provide high clock speeds, and ECS instance types that use local disks based on your business requirements. For more information, see Specify an ECS instance type to create an elastic container instance.

Configuration example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: vk-cos-use
  labels:
    app: cos
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cos
  template:
    metadata:
        annotations: 
            "k8s.aliyun.com/eci-use-specs": "2-4Gi,4-8Gi,ecs.c6.xlarge" # You can specify multiple specifications. The specifications can be vCPUs and memory, or specific ECS instance types. 
        labels:
            app: cos
    spec:
      containers:
      - name: u1
        image: "registry-vpc.cn-beijing.aliyuncs.com/lxx/cos-4g"
      nodeName: virtual-kubelet

Create a preemptible elastic container instance

You can run stateless applications and jobs on preemptible instances to reduce costs. You can add annotations to create a preemptible instance. For more information, see Create a preemptible instance.

Configuration example:

apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
kind: Deployment
metadata:
  name: nginx-deployment-basic
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
      annotations:
        k8s.aliyun.com/eci-use-specs : "ecs.c5.large"  # Specify an ECS instance type. 
        k8s.aliyun.com/eci-spot-strategy: "SpotWithPriceLimit"  # Use the custom strategy to specify the maximum price. 
        k8s.aliyun.com/eci-spot-price-limit: "0.250"   # Specify the maximum price per hour for the preemptible instance. 
    spec:
    #  nodeSelector:
    #    env: test-team
      containers:
      - name: nginx
        image: nginx:1.7.9 # replace it with your exactly <image_name:tags>
        ports:
        - containerPort: 80

Customize CPU options

The CPU options of an elastic container instance include the number of physical CPU cores and the number of threads per core. An elastic container instance may support custom CPU options. This is determined by the system based on how the instance is created. For more information, see Customize CPU options.

Configuration example:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.aliyun.com/eci-use-specs : "ecs.c6.2xlarge"    # Specify an ECS instance type that supports custom CPU options. 
    k8s.aliyun.com/eci-cpu-option-core: 2                 # Set the number of physical CPU cores to 2. 
    k8s.aliyun.com/eci-cpu-option-ht: 1                    # Set the number of threads per core to 1. This value indicates that the HT is disabled. 
  name: nginx-test
spec:
  containers:
  - name: nginx
    image: nginx:latest
  restartpolicy: Always

Configure rescheduling for elastic container instances

Pods may fail to be scheduled to virtual nodes. You can add an annotation to enable rescheduling for pods. This ensures that the system continues to retry to schedule pods instead of returning failures even if the asynchronous scheduling fails.

Configuration example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: set-eci
  labels:
    app: vk
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
        annotations: 
            k8s.aliyun.com/eci-reschedule-enable: "true"    # Enable rescheduling for elastic container instances. 
        labels:
            app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
      nodeName: virtual-kubelet

Change the state of the pods that cannot be created to Failed.

By default, if an error occurs when a pod is being created, the system retries for up to the specified maximum number of retries. If the pod cannot be created after the maximum number of retries, the pod enters the Pending state. For some jobs, you may want the pod to enter the Failed state. In these cases, you can add an annotation to change the state of a pod that cannot be created to Failed:

Configuration example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: set-pod-fail-on-create-err
  labels:
    app: vk
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
        annotations: 
            k8s.aliyun.com/pod-fail-on-create-err: "true"  # Set the status to Failed if the pod fails to be created. 
        labels:
            app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
      nodeName: virtual-kubelet

Use an image cache as a custom resource definition (CRD) to accelerate pod creation

You can use image caches to accelerate pod creation. You can add annotations to specify image caches or enable automatic matching for image caches. For more information, see Use an image cache CRD to accelerate pod creation.

Configuration example:

  • Specify an image cache

    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        k8s.aliyun.com/eci-image-snapshot-id: imc-2ze5tm5gehgtiiga****   # Specify an image cache. 
      name: nginx-imagecache-id
    spec:
      containers:
      - image: nginx:1.7.9
        imagePullPolicy: IfNotPresent
        name: nginx
        resources:
          limits:
            cpu: 300m
            memory: 200Mi
          requests:
            cpu: 200m
            memory: 100Mi
      nodeName: virtual-kubelet
  • Automatic matching

    apiVersion: v1
    kind: Pod
    metadata:
      annotations:
        k8s.aliyun.com/eci-image-cache: "true"    # Specify whether to enable automatic matching for image caches. 
      name: nginx-auto-match
    spec:
      containers:
      - image: nginx:1.7.9
        imagePullPolicy: IfNotPresent
        name: nginx
        resources:
          limits:
            cpu: 300m
            memory: 200Mi
          requests:
            cpu: 200m
            memory: 100Mi
      nodeName: virtual-kubelet

Specify a Container Registry Enterprise Edition instance

Container Registry allows you to pull images without the need to enter a password. You can add annotations to specify a Container Registry Enterprise Edition instance and pull images from an image repository in the instance. For more information, see Configure password-free settings to pull images from Container Registry Enterprise Edition instances.

Configuration example:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    k8s.aliyun.com/acr-instance-id: cri-j36zhodptmyq****      # Specify the ID of a Container Registry Enterprise Edition instance. 
  name: cri-test
spec:
  containers:
  - image: test****-registry.cn-beijing.cr.aliyuncs.com/eci_test/nginx:1.0   # Pull images over the Internet. 
    imagePullPolicy: Always
    name: nginx
  restartPolicy: Never

Associate an EIP with a pod

If you want to connect a pod over the Internet, you can associate an EIP with the pod. For more information, see Bind an EIP to a pod.

Configuration example:

  • Automatically create and associate an EIP

    apiVersion: v1
    kind: Pod
    metadata:
      name: nginx
      annotations:
        k8s.aliyun.com/eci-with-eip: "true"    # Enable automatic EIP creation. 
        k8s.aliyun.com/eip-bandwidth: "10"     # Specify the bandwidth. Default value: 5. Unit: Mbit/s. 
    spec:
      containers:
      - image: registry-vpc.cn-hangzhou.aliyuncs.com/jovi/nginx:alpine
        imagePullPolicy: Always
        name: nginx
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
      restartPolicy: OnFailure
  • Associate an existing EIP

    apiVersion: v1
    kind: Pod
    metadata:
      name: nginx
      annotations:
        k8s.aliyun.com/eci-eip-instanceid: "eip-bp1q5n8cq4p7f6dzu****"   # Specify the EIP. 
    spec:
      containers:
      - image: registry-vpc.cn-hangzhou.aliyuncs.com/jovi/nginx:alpine
        imagePullPolicy: Always
        name: nginx
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
      restartPolicy: OnFailure

Configure IPv6

Compared with IPv4 addresses, the number of IPv6 addresses is sufficient to connect more devices to the Internet. You can add annotations to allocate IPv6 addresses to pods. For more information, see Assign an IPv6 address to an elastic container instance.

Configuration example:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
  annotations:
    k8s.aliyun.com/eci-enable-ipv6: "true"    # Enable automatic IPv6 assignation. 
spec:
  containers:
  - name: nginx
    image: nginx
  nodeName: virtual-kubelet

Configure inbound and outbound bandwidths

You can configure inbound and outbound bandwidths for elastic container instances. You can add annotations to limit the inbound and outbound bandwidths of pods. For more information, see Limit the bandwidth of an elastic container instance.

Configuration example:

apiVersion: v1
kind: Pod
metadata:
  name: eci-qos
  annotations:
    kubernetes.io/ingress-bandwidth: 40M    # Specify the inbound bandwidth. 
    kubernetes.io/egress-bandwidth: 10M    # Specify the outbound bandwidth. 
spec:
  containers:
  - name: nginx
    image: nginx:latest
    command: ["bash","-c","sleep 100000"]

Specify the size of the temporary storage space

Each elastic container instance provides 20 GiB storage. If this storage space is insufficient, you can create a temporary storage space. You can add an annotation to configure the size of the temporary storage space. For more information, see Create a custom temporary storage space.

Configuration example:

apiVersion: v1
kind: Pod
metadata:
  name: test
  annotations:
    k8s.aliyun.com/eci-extra-ephemeral-storage: "50Gi"# Specify the size of temporary storage space. 
spec:
  containers:
  - name: nginx
    image: nginx:latest
    imagePullPolicy: IfNotPresent
  restartPolicy: Always
  nodeName: virtual-kubelet

Configure a directory to store core dump files

If a program unexpectedly terminates or exits, a core dump file is generated. By default, this file is named core.pid and stored in the directory that is being used. You can add an annotation to configure a directory to store core dump files. For more information, see View core dump files.

Configuration example:

apiVersion: v1
kind: Pod
metadata:
  name: test
  annotations:
    k8s.aliyun.com/eci-core-pattern: "pod/data/dump/core"  # Specify the directory in which the core dump files are stored. 
spec:
  containers:
  - image: nginx:latest
    name: test-container
    volumeMounts:
    - mountPath: /pod/data/dump/
      name: default-volume
  volumes:
  - name: nfs
    nfs:
      server: 143b24****-gfn3.cn-beijing.nas.aliyuncs.com
      path: /dump/
      readOnly: false

Configure the NTP service

You can add the k8s.aliyun.com/eci-ntp-server annotation to configure the NTP service for pods. For more information, see Configure an NTP server for pods.

Configuration example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: set-ngnix-ntp
  labels:
    app: vk
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
        annotations: 
            k8s.aliyun.com/eci-ntp-server: 100.100.5.*,100.100.5.*  # Specify the IP addresses of your NTP servers. 
        labels:
            app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
      nodeName: virtual-kubelet