Configure IPv6 internet bandwidth and egress-only rules for a pod using annotations - Container Service for Kubernetes

How it works

Each pod in a dual-stack ACK cluster receives an IPv6 address from its VPC. When you add the k8s.aliyun.com/ipv6-bandwidth annotation to a pod, the ipv6gw controller calls the Alibaba Cloud OpenAPI to allocate an IPv6 Internet bandwidth plan and associate it with that address. Inbound and outbound IPv6 traffic is then routed directly through the IPv6 gateway.

To allow outbound-only traffic, add the k8s.aliyun.com/ipv6-egress-only annotation. This creates an egress-only rule on the IPv6 gateway.

Prerequisites

Before you begin, ensure that you have:

An ACK managed cluster or ACK dedicated cluster with IPv6 dual-stack enabled and the Terway network plugin installed. For setup instructions, see Create an ACK managed cluster and What is an IPv6 gateway?
An IPv6 gateway created in your VPC and the ipv6gw controller deployed. See Create and manage an IPv6 gateway

Important

Only some ECS instance types support IPv6. Verify that your nodes use a supported instance type before proceeding. See Instance families.

Limitations

Standard ECS nodes only: The ack-extend-network-controller component supports standard ECS nodes and pods only. It does not support ECI or ACS instances. For those instance types, see Assign IPv6 addresses to ECI pods and Assign IPv6 addresses to ACS pods.
IPv6 gateway quotas apply: Bandwidth limits and the number of configuration entries are subject to IPv6 gateway quotas. See Limits.

Enable IPv6 gateway features

Step 1: Configure RAM permissions

Grant the Worker RAM role the permissions required by the ipv6gw controller.

Log on to the Container Service Management Console. In the left navigation pane, click Clusters.
Click the name of your cluster. In the left navigation pane, click Cluster Information.
On the Basic Information tab, click the link next to Worker RAM Role.

Create a custom policy with the following content. For instructions, see Step 1: Create a custom policy.

{
  "Effect": "Allow",
  "Action": [
    "vpc:DescribeIpv6Addresses",
    "vpc:CreateIpv6EgressOnlyRule",
    "vpc:DeleteIpv6EgressOnlyRule",
    "vpc:DescribeIpv6EgressOnlyRules",
    "vpc:AllocateIpv6InternetBandwidth",
    "vpc:DeleteIpv6InternetBandwidth"
  ],
  "Resource": ["*"],
  "Condition": {}
}

Attach the custom policy to the Worker RAM role. See Step 2: Grant permissions to the Worker RAM role of your cluster.

Step 2: Install and configure the ack-extend-network-controller component

Install ack-extend-network-controller from the ACK Marketplace and enable the ipv6gw controller. For installation instructions, see Marketplace.

Use the following configuration when installing the component. Replace the placeholder values with your actual cluster ID, region, and VPC ID.

clusterID: "c11ba338192xxxxxxx"          # Your cluster ID
regionID: "cn-hangzhou"                  # Your region ID
vpcID: "vpc-bp1rkq0zxxxxxx"              # Your VPC ID
enableControllers:
  - ipv6gw                               # Enable the IPv6 gateway controller
networkController:
  ipv6GatewayController:
    maxConcurrentReconciles: 10

credential:                              # For ACK managed and dedicated clusters, use RAM roles instead of AccessKey pairs
  accessKey: ""
  accessSecret: ""

Pod annotations

Use the following annotations to configure IPv6 Internet bandwidth and egress behavior on a pod.

Annotation	Description
`k8s.aliyun.com/ipv6-bandwidth`	Public bandwidth in Mbps. Valid values: 1 to 5000. See AllocateIpv6InternetBandwidth.
`k8s.aliyun.com/ipv6-internet-charge-type`	Billing method. `PayByBandwidth` (default) or `PayByTraffic`. See AllocateIpv6InternetBandwidth.
`k8s.aliyun.com/ipv6-egress-only`	Set to an empty string (`""`) to create an egress-only rule.

Configure IPv6 bandwidth for a pod

The following example creates a Deployment with 10 Mbps IPv6 bandwidth and an egress-only rule.

Console procedure

Log on to the Container Service Management Console. In the left navigation pane, click Clusters.
Click the name of your cluster. In the left navigation pane, click Workloads > Deployments.

In the upper-right corner, click Create from YAML. Paste the following YAML into the Templates editor, then click Create.

This example configures 10 Mbps IPv6 bandwidth for the pod and creates an egress-only rule.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example
  labels:
    app: example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
      annotations:
        k8s.aliyun.com/ipv6-bandwidth: "10"  # Public bandwidth, in Mbps
        k8s.aliyun.com/ipv6-egress-only: ""  # Create an egress-only rule
    spec:
      containers:
      - name: example
        image: nginx

Expected output:

deployment.apps/example created

On the Deployments page, click the deployment named example. On the example overview page, find the name of the pod that is created.

kubectl procedure

Create a file named example.yaml with the following content.

This example configures 10 Mbps IPv6 bandwidth for the pod and creates an egress-only rule.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example
  labels:
    app: example
spec:
  replicas: 1
  selector:
    matchLabels:
      app: example
  template:
    metadata:
      labels:
        app: example
      annotations:
        k8s.aliyun.com/ipv6-bandwidth: "10"  # Public bandwidth, in Mbps
        k8s.aliyun.com/ipv6-egress-only: ""  # Create an egress-only rule
    spec:
      containers:
      - name: example
        image: nginx

Run kubectl apply -f example.yaml.

Verify the configuration

After the Deployment is created, verify that the bandwidth plan and egress-only rule are configured correctly.

List the PodIPv6GW resources in the namespace where the Deployment was created:
```
kubectl get podipv6gws -n <namespace>
```

Inspect the PodIPv6GW resource that was automatically created for the pod:

kubectl get podipv6gws.alibabacloud.com -oyaml example-674b897446-q****

Expected output:

apiVersion: alibabacloud.com/v1beta1
kind: PodIPv6GW
metadata:
  creationTimestamp: "2024-02-07T06:00:48Z"
  finalizers:
  - ipv6gw-controller.alibabacloud.com/finalizer
  generation: 1
  name: example-674b897446-q****
  namespace: default
  ownerReferences:
  - apiVersion: v1
    kind: Pod
    name: example-674b897446-q****
    uid: bdf86c45-7**5-4**4-b**3-9b****
  resourceVersion: "11488"
  uid: 78747d60-9**8-4**e-b**8-f7f7c****
spec:
  bandwidth: 10
  egressOnlyRule: true
  ipv6Address: 2408:4006:1115:xxxx:xxxx:xxxx:xxxx:xxxx
status:
  ipv6AddressID: ipv6-xxx           # The IPv6 address resource ID
  ipv6EgressOnlyRuleID: ipv6py-xxx  # The egress-only rule resource ID
  ipv6GatewayID: ipv6gw-xxx         # The IPv6 gateway resource ID
  ipv6InternetBandwidthID: ipv6bw-xxx  # The bandwidth plan resource ID

The status section confirms the IPv6 gateway, bandwidth plan, and egress-only rule were created successfully.

FAQ

Can I deploy ack-extend-network-controller in an ACK serverless cluster?

This component is not designed for ACK serverless clusters. If your use case requires it, generate an AccessKey pair for a RAM user first — see Create a RAM user and Create a custom policy — then enter the AccessKey pair in the corresponding fields when deploying the component from the Marketplace.

What's next

Notes on accessing external networks from pods