Configure a static IP address for a pod - Container Compute Service

When you upgrade applications that are sensitive to IP address changes, reconstructing a pod changes its IP address, which can disrupt business operations. Alibaba Cloud Container Service (ACS) lets you keep pod IP addresses unchanged after they are reconstructed. This feature supports application upgrades in scenarios where static IP addresses are required.

Scope

This feature is available only for general-purpose and compute-optimized CPU pods.
Static IP addresses cannot be retained across ECS nodes and ACS virtual nodes.
If a pod is first scheduled to an ECS node and then rescheduled to an ACS virtual node during reconstruction, the static IP address cannot be retained. The same applies if the pod is moved from an ACS virtual node to an ECS node.

Feature description

In a Kubernetes cluster, workloads such as StatefulSet are often used to manage stateful applications. During an application upgrade, the workload performs a rolling upgrade by deleting old pods and creating new ones. For most services that use a microservices model, this method meets the upgrade requirements. However, because reconstructing a pod changes its IP address, some IP-sensitive applications cannot be directly upgraded. Examples include the following:

Perform whitelist verification on the source IP address of the pod. For example, the source IP address of the pod is authorized by a database.
Service discovery and route allocation are implemented based on pod IP addresses.

ACS provides a feature for stateful applications that keeps the IP address of a pod unchanged after it is reconstructed. When a pod is released, the system retains its static IP address for a specified period. This feature relies on the stable and predictable pod names provided by StatefulSet, such as web-0 and web-1. If the static IP address feature is enabled, the system assigns the retained IP address to a new pod that has the same cluster ID, namespace, and pod name as a released pod. This ensures that the IP addresses of pods in the same StatefulSet remain unchanged.

The old pod continues to use the IP address until it is completely deleted. This may slow down the creation of the new pod.

Configuration description

You can add annotations to the pod's metadata to enable the static IP address feature and specify the retention period for the idle IP address. The following table describes the annotations.

Annotation

Example value

Description

network.alibabacloud.com/enable-fixed-ip

"true"

Valid values:

true: Enables the static IP address feature for the pod. The default retention period is 48 hours.
false or if the annotation is not configured: The pod does not use a static IP address.

network.alibabacloud.com/fixed-ip-release-after

"48"

The retention period for an idle static IP address. This is the duration for which the static IP address is retained after the pod that uses it is released. The unit is hours.

This annotation takes effect only when network.alibabacloud.com/enable-fixed-ip: "true" is also configured.

The retention period of the static IP address starts from the time you delete the pod. If you repeatedly delete and create the pod, the time of the last deletion is used.

Usage example

Create a file named fixed-ip.yaml with the following content. Then, run the kubectl apply -f fixed-ip.yaml command to create the sample application.

This example creates a StatefulSet that contains two pods. The pods are configured with the network.alibabacloud.com/enable-fixed-ip: "true" annotation to enable the static IP address feature.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: web
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2
  template:
    metadata:
      name: nginx
      labels:
        app: nginx
      annotations:
        network.alibabacloud.com/enable-fixed-ip: "true"
    spec:
      containers:
      - name: nginx
        image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
        ports:
        - containerPort: 80

Monitor the pod status changes.

kubectl get pod -o wide -w

Expected output:

After the pods are created, the IP addresses of the two pods are displayed. For example, web-0 has the IP address 192.168.1.8 and web-1 has the IP address 192.168.2.2.

NAME    READY   STATUS              RESTARTS   AGE   IP            NODE                              NOMINATED NODE   READINESS GATES
web-0   0/1     Pending             0          1s    <none>        <none>                            <none>           <none>
web-0   0/1     Pending             0          20s   <none>        <none>                            <none>           <none>
web-0   0/1     Pending             0          20s   <none>        virtual-kubelet-cn-wulanchabu-b   <none>           <none>
web-0   0/1     ContainerCreating   0          24s   192.168.1.8   virtual-kubelet-cn-wulanchabu-b   <none>           <none>
web-0   1/1     Running             0          25s   192.168.1.8   virtual-kubelet-cn-wulanchabu-b   <none>           <none>
web-1   0/1     Pending             0          0s    <none>        <none>                            <none>           <none>
web-1   0/1     Pending             0          21s   <none>        <none>                            <none>           <none>
web-1   0/1     Pending             0          21s   <none>        virtual-kubelet-cn-wulanchabu-c   <none>           <none>
web-1   0/1     ContainerCreating   0          24s   192.168.2.2   virtual-kubelet-cn-wulanchabu-c   <none>           <none>
web-1   1/1     Running             0          25s   192.168.2.2   virtual-kubelet-cn-wulanchabu-c   <none>           <none>

Open a new command-line terminal and perform a rolling upgrade on the sample application.
```
kubectl rollout restart statefulset web
```

Switch back to the previous terminal to observe the pod status changes during the rolling upgrade.

The expected output is as follows. You can see that after the old pods are deleted, the StatefulSet automatically creates new pods. The IP addresses of the new pods remain 192.168.1.8 and 192.168.2.2.

NAME    READY   STATUS              RESTARTS   AGE   IP            NODE                              NOMINATED NODE   READINESS GATES
...
web-1   1/1     Terminating         0          25m   192.168.2.2   virtual-kubelet-cn-wulanchabu-c   <none>           <none>
web-1   0/1     Completed           0          25m   192.168.2.2   virtual-kubelet-cn-wulanchabu-c   <none>           <none>
web-1   0/1     Pending             0          0s    <none>        <none>                            <none>           <none>
web-1   0/1     Pending             0          1s    <none>        <none>                            <none>           <none>
web-1   0/1     Pending             0          17s   <none>        <none>                            <none>           <none>
web-1   0/1     Pending             0          17s   <none>        virtual-kubelet-cn-wulanchabu-c   <none>           <none>
web-1   0/1     Pending             0          17s   <none>        virtual-kubelet-cn-wulanchabu-c   <none>           <none>
web-1   0/1     ContainerCreating   0          21s   192.168.2.2   virtual-kubelet-cn-wulanchabu-c   <none>           <none>
web-1   1/1     Running             0          22s   192.168.2.2   virtual-kubelet-cn-wulanchabu-c   <none>           <none>
web-0   1/1     Terminating         0          26m   192.168.1.8   virtual-kubelet-cn-wulanchabu-b   <none>           <none>
web-0   0/1     Completed           0          26m   192.168.1.8   virtual-kubelet-cn-wulanchabu-b   <none>           <none>
web-0   0/1     Pending             0          0s    <none>        <none>                            <none>           <none>
web-0   0/1     Pending             0          1s    <none>        <none>                            <none>           <none>
web-0   0/1     Pending             0          20s   <none>        <none>                            <none>           <none>
web-0   0/1     Pending             0          20s   <none>        virtual-kubelet-cn-wulanchabu-b   <none>           <none>
web-0   0/1     Pending             0          20s   <none>        virtual-kubelet-cn-wulanchabu-b   <none>           <none>
web-0   0/1     ContainerCreating   0          24s   192.168.1.8   virtual-kubelet-cn-wulanchabu-b   <none>           <none>
web-0   1/1     Running             0          24s   192.168.1.8   virtual-kubelet-cn-wulanchabu-b   <none>           <none>