Container Service for Kubernetes:Attach an independent EIP to a pod

ack-extend-network-controller (recommended)

1. Create a custom policy with the following content. For more information, see Step 1: Create a custom policy.

      {
        "Effect": "Allow",
        "Action": [
          "vpc:DescribeVSwitches",
          "vpc:AllocateEipAddress",
          "vpc:AllocateEipAddressPro",
          "vpc:DescribeEipAddresses",
          "vpc:AssociateEipAddress",
          "vpc:UnassociateEipAddress",
          "vpc:ReleaseEipAddress",
          "vpc:AddCommonBandwidthPackageIp",
          "vpc:RemoveCommonBandwidthPackageIp",
          "vpc:TagResources",
          "ecs:DescribeNetworkInterfaces"
        ],
        "Resource": [
          "*"
        ],
        "Condition": {}
      }

2. Grant permissions to the worker RAM role of the cluster. For more information, see Step 2: Grant permissions to the Worker RAM role of the cluster.

Terway (legacy)

Terway requires EIP-related permissions to allocate and configure EIPs for pods.

ACK managed cluster or ACK managed cluster of Basic Edition created in or after June 2020

1. Create a custom policy with the following content. For more information, see Step 1: Create a custom policy.

      {
        "Effect": "Allow",
        "Action": [
          "vpc:DescribeVSwitches",
          "vpc:AllocateEipAddress",
          "vpc:DescribeEipAddresses",
          "vpc:AssociateEipAddress",
          "vpc:UnassociateEipAddress",
          "vpc:ReleaseEipAddress",
          "vpc:AddCommonBandwidthPackageIp",
          "vpc:RemoveCommonBandwidthPackageIp"
        ],
        "Resource": [
          "*"
        ],
        "Condition": {}
      }

2. Grant permissions to the worker RAM role of the cluster. For more information, see Step 2: Grant permissions to the Worker RAM role of the cluster.

ACK dedicated cluster or ACK managed cluster created before June 2020

1. Create a custom policy with the same content as above. For more information, see Step 1: Create a custom policy.

2. On the AliyunCSManagedNetworkRole RAM role page, go to the Permissions tab and click Add Permissions. Then, click Custom Policy, select the custom policy that you created in the previous step, and click OK.

Install ack-extend-network-controller (recommended)

1. Log on to the ACK console. In the left-side navigation pane, choose Marketplace > Marketplace.

2. On the App Marketplace page, enter ack-extend-network-controller in the search box and click the application.

3. On the application details page, click Deploy in the upper-right corner.

4. In the Create panel, select the cluster and namespace, and then click Next.

5. On the parameter configuration page, select a version number, set the parameters, and then click OK. The following table describes the parameters. Sample configuration:

   Parameter	Type	Required	Description
   clusterID	string	Yes	The cluster ID.
   regionID	string	Yes	The ID of the region where the cluster is deployed.
   enableControllers	[]string	Yes	Set to eip to enable the EIP feature.
   networkController.vpcid	string	Yes	The VPC ID of the cluster.
   networkController.eipController.maxConcurrentReconciles	int	No	The number of concurrent EIP controllers.
   networkController.eipController.garbageCollectionPeriodInMinutes	int	No	The period at which the EIP controller cleans up fixed EIPs.
   customStatefulWorkloadKinds	[]string	No	The Kind of the custom stateful controller.
   enableVirtualNode	bool	No	Specifies whether to support Elastic Container Instance (ECI) virtual nodes. Important If enabled, the controller allocates EIPs to pods of ECI instances. The pod annotation must not conflict with the EIP feature supported by ECI. Use with caution.
   enableRRSA	bool	No	Authenticates requests using RAM Roles for Service Accounts (RRSA). Note Supported by v0.10.0 and later. Only the ack-pod-identity-webhook injection method is supported. Configure the corresponding label for the kube-system namespace. For more information, see Isolate pod permissions based on RRSA.
   rrsaRoleName	string	No	The RRSA role name. Note Supported by v0.10.0 and later.
   credential.accessKey	string	No	The AccessKey ID of the account with EIP permissions. Not required if you completed Step 1. Important This stores sensitive information in a Kubernetes Secret. Not recommended.
   credential.accessSecret	string	No	The AccessKey secret of the account with EIP permissions. Not required if you completed Step 1. Important This stores sensitive information in a Kubernetes Secret. Not recommended.
   kubeClientQPS	float32	No	The queries per second (QPS) limit for the component to access the cluster API server. Note Supported by v0.12.2 and later.
   kubeClientBurst	int	No	The maximum number of burst requests for the component to access the cluster API server. Note Supported by v0.12.2 and later.

      clusterID: "c11ba338192xxxxxxx"
      regionID: "cn-hangzhou"
      vpcID: "vpc-bp1rkq0zxxxxxx"
      enableControllers:
        - eip
   
      networkController:
        eipController:
          maxConcurrentReconciles: 1
          garbageCollectionPeriodInMinutes: 1
        customStatefulWorkloadKinds:
        - foo
   
      credential:
        accessKey: ""
        accessSecret: ""

To update the version and parameters of ack-extend-network-controller, see Modify a Helm chart.

Upgrade Terway (legacy)

Make sure the cluster uses the Terway plugin, and then upgrade terway-eni or terway-eniip to a version that supports the EIP feature. For more information, see Manage components.

The Terway plugin version must be v1.0.10.280-gdc2cb6c-aliyun or later, and earlier than v1.7.0. For more information about Terway versions, see Terway.

Automatically allocate an EIP

Add annotations to the pod template to create and attach an EIP when a pod starts.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example
  labels:
    app: example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
      annotations:
        k8s.aliyun.com/pod-with-eip: "true"
        k8s.aliyun.com/eip-bandwidth: "5"
    spec:
      readinessGates:
      - conditionType: "k8s.aliyun.com/eip"
      containers:
      - name: example
        image: nginx

kubectl get podeip -n <namespace> <podname> -o yaml

apiVersion: alibabacloud.com/v1beta1
kind: PodEIP
metadata:
  creationTimestamp: "2023-12-15T04:25:37Z"
  finalizers:
  - podeip-controller.alibabacloud.com/finalizer
  generation: 1
  name: example-xxx
  namespace: default
  resourceVersion: "222800"
  uid: 43xxx-f1xx-4xxx-b3xx-969faxxx
spec:
  allocationID: eip-2ze2qe8zsxxx
  allocationType:
    releaseStrategy: Follow
    type: Auto
status:
  eipAddress: 39.102.XX.XX
  internetChargeType: PayByTraffic
  isp: BGP
  networkInterfaceID: eni-2zeagv8f3xxxx
  podLastSeen: "2023-12-15T05:18:47Z"
  privateIPAddress: 192.168.XX.XX
  resourceGroupID: rg-acfmwxxxxxsq
  status: InUse

Specify an existing EIP

Attach an existing EIP by specifying its instance ID in a pod annotation. This annotation does not modify the EIP instance configuration; it only attaches the EIP to the pod.

Keep the same EIP after a pod restart (fixed EIP)

A fixed EIP makes sure that a pod continues to use the same EIP address after it is rebuilt. Combine this with automatic EIP allocation for stateful applications.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: example
  labels:
    app: example
spec:
  serviceName: "example"
  replicas: 2
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
      annotations:
        k8s.aliyun.com/pod-with-eip: "true"
        k8s.aliyun.com/pod-eip-release-strategy: "10m"
    spec:
      containers:
      - name: example
        image: nginx

kubectl get podeip -n <namespace> -o yaml

apiVersion: v1
items:
- apiVersion: alibabacloud.com/v1beta1
  kind: PodEIP
  metadata:
    creationTimestamp: "2023-12-15T03:28:01Z"
    finalizers
    - podeip-controller.alibabacloud.com/finalizer
    generation: 1
    name: example-0
    namespace: default
    resourceVersion: "227221"
    uid: 79954xx-17xx-4dxx-b7xx-15b84xxx
  spec:
    allocationID: eip-2ze08metxxx
    allocationType:
      releaseAfter: 10m
      releaseStrategy: TTL
      type: Auto
  status:
    eipAddress: 39.105.XX.XX
    internetChargeType: PayByTraffic
    isp: BGP
    networkInterfaceID: eni-2ze4tkg4xxx
    podLastSeen: "2023-12-15T05:31:34Z"
    privateIPAddress: 192.168.XX.XX
    resourceGroupID: rg-acfmwxxx
    status: InUse
- apiVersion: alibabacloud.com/v1beta1
  kind: PodEIP
  metadata:
    creationTimestamp: "2023-12-15T03:28:03Z"
    finalizers:
    - podeip-controller.alibabacloud.com/finalizer
    generation: 1
    name: example-1
    namespace: default
    resourceVersion: "227222"
    uid: 1339xxxe7-97xx-46xx-9bxx-537690xxx
  spec:
    allocationID: eip-2zetwhffqxxx
    allocationType:
      releaseAfter: 10m
      releaseStrategy: TTL
      type: Auto
  status:
    eipAddress: 39.105.XX.XX
    internetChargeType: PayByTraffic
    isp: BGP
    networkInterfaceID: eni-2zeagv8f3wxxx
    podLastSeen: "2023-12-15T05:31:34Z"
    privateIPAddress: 192.168.XX.XX
    resourceGroupID: rg-acfmwqnwxxx
    status: InUse
- apiVersion: alibabacloud.com/v1beta1
  kind: PodEIP
  metadata:
    creationTimestamp: "2023-12-15T04:25:37Z"
    finalizers:
    - podeip-controller.alibabacloud.com/finalizer
    generation: 1
    name: example-5bxxx-9xx
    namespace: default
    resourceVersion: "227220"
    uid: 43cdfxxx-f1xx-42xx-b3xx-969fxxx
  spec:
    allocationID: eip-2ze2qe8zsmnxxx
    allocationType:
      releaseStrategy: Follow
      type: Auto
  status:
    eipAddress: 39.102.XX.XX
    internetChargeType: PayByTraffic
    isp: BGP
    networkInterfaceID: eni-2zeagv8f3wxxx
    podLastSeen: "2023-12-15T05:31:34Z"
    privateIPAddress: 192.168.XX.XX
    publicIpAddressPoolID: pippool-2ze498cxxx
    resourceGroupID: rg-acfmwqnxxx
    status: InUse
kind: List
metadata:
  resourceVersion: ""

Enable EIP in Terway (legacy)

1. Modify the Terway ConfigMap to enable the EIP pool:

      kubectl edit cm eni-config -n kube-system

2. Add the following entry to eni_conf:

   To forcibly detach a previously attached EIP when specifying an EIP instance, also add "allow_eip_rob": "true" to eni_conf.

      "enable_eip_pool": "true"

3. After modifying the file, press the Esc key, enter :wq!, and press the Enter key to save and exit.

4. Recreate the Terway pods:

   If you installed terway-eni, replace terway-eniip with terway-eni.

      kubectl delete pod -n kube-system -l app=terway-eniip

5. Add annotations to the pod template. Automatic EIP allocation: Specify an existing EIP:

   Important

   - A single EIP cannot be associated with multiple pods. This is not applicable to Deployments or StatefulSets with multiple replicas. - By default, if the EIP is already attached to an instance, it cannot be attached to a pod. To detach the EIP from the previous instance first, configure "allow_eip_rob": "true" in the ConfigMap. - Specify an EIP instance ID only for single-replica instances.

      apiVersion: apps/v1
      kind: Deployment
      metadata:
        name: nginx-deployment-basic
        labels:
          app: nginx
      spec:
        replicas: 2
        selector:
          matchLabels:
            app: nginx
        template:
          metadata:
            annotations:
              network.alibabacloud.com/pod-with-eip: "true"   # Automatically allocate a public EIP
              network.alibabacloud.com/eip-bandwidth: "5"      # Bandwidth: 5 Mbit/s (default)
            labels:
              app: nginx
          spec:
            containers:
            - name: nginx
              image: nginx

      k8s.aliyun.com/pod-eip-instanceid: "<your-eip-instance-id>"