When you create a Container Service for Kubernetes (ACK) cluster, you must specify a virtual private cloud (VPC), vSwitches, the CIDR block of pods, and the CIDR block of Services. Therefore, we recommend that you plan the IP address of each Elastic Compute Service (ECS) instance, the CIDR block of pods, and the CIDR block of Services before you create an ACK cluster. This topic describes how to plan CIDR blocks for an ACK cluster deployed in a VPC and how each CIDR block is used.

VPC-related CIDR blocks and cluster-related CIDR blocks

Before you create a VPC, you must plan the CIDR block of the VPC and the CIDR blocks of vSwitches in the VPC. Before you create an ACK cluster, you must plan the CIDR block of pods and the CIDR block of Services. ACK supports the Terway and Flannel plug-ins. The following figures show the network architectures of ACK clusters that use Terway and Flannel.
Figure 1. Terway
terway
Figure 2. Flannel
Flannel

Precautions

To install Terway or Flannel in an ACK cluster, you must specify CIDR blocks and other parameters as described in the following table.
Parameter Terway Flannel
VPC When you create a VPC, you must select a CIDR block for the VPC. The valid VPC CIDR blocks are 10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0/16.

IPv6 CIDR blocks are assigned by the VPC after you enable IPv6 for the VPC. If you want to enable IPv6 for containers, select Terway as the network plug-in.

vSwitch The IP addresses of ECS instances are assigned from vSwitches. This allows nodes to communicate with each other. The CIDR blocks that you specify when you create vSwitches in the VPC must be subsets of the VPC CIDR block. This means that the vSwitch CIDR blocks must be the same as or fall within the VPC CIDR block. When you set this parameter, take note of the following items:
  • You must select vSwitches that belong to the VPC where the cluster resides.
  • IP addresses from the CIDR block of a vSwitch are allocated to the ECS instances that are attached to the vSwitch.
  • You can create multiple vSwitches in a VPC. However, the CIDR blocks of these vSwitches cannot overlap with each other.
  • You must specify a pod vSwitch for each vSwitch in the same zone. For more information about zones, see Regions and zones.
The IP addresses of ECS instances are assigned from vSwitches. This allows nodes to communicate with each other. The CIDR blocks that you specify when you create vSwitches in the VPC must be subsets of the VPC CIDR block. This means that the vSwitch CIDR blocks must be the same as or fall within the VPC CIDR block. When you set this parameter, take note of the following items:
  • You must select vSwitches that belong to the VPC where the cluster resides.
  • IP addresses from the CIDR block of a vSwitch are allocated to the ECS instances that are attached to the vSwitch.
  • You can create multiple vSwitches in a VPC. However, the CIDR blocks of these vSwitches cannot overlap with each other.
Pod vSwitch The IP addresses of pods are assigned from the CIDR block of pod vSwitches. This allows pods to communicate with each other. A pod is a group of containers in a Kubernetes cluster. Each pod has an IP address. The CIDR blocks that you specify when you create pod vSwitches in the VPC must be subsets of the VPC CIDR block. When you set this parameter, take note of the following items:
  • You must select vSwitches that belong to the VPC where the cluster resides.
  • In an ACK cluster that has Terway installed, the IP addresses of pods are assigned by pod vSwitches.
  • The CIDR blocks of pod vSwitches cannot overlap with the CIDR block specified by Service CIDR.
  • You must specify a pod vSwitch for each vSwitch in the same zone. For more information about zones, see Regions and zones.
You do not need to set this parameter if you install Flannel.
Pod CIDR Block You do not need to set this parameter if you install Terway. The IP addresses of pods are allocated from the pod CIDR block. This allows pods to communicate with each other. A pod is a group of containers in a Kubernetes cluster. Each pod has an IP address. When you set this parameter, take note of the following items:
  • Enter a CIDR block in the Pod CIDR Block field.
  • The CIDR block of pods cannot overlap with the CIDR blocks of vSwitches.
  • The CIDR block of pods cannot overlap with the CIDR block specified by Service CIDR.

For example, if the VPC CIDR block is 172.16.0.0/12, the CIDR block of pods cannot be 172.16.0.0/16 or 172.17.0.0/16, because these CIDR blocks are subsets of 172.16.0.0/12.

Service CIDR The CIDR block of Services. Service is an abstraction in Kubernetes. Each ClusterIP Service has an IP address. When you set this parameter, take note of the following items:
  • The IP address of a Service is effective only within the Kubernetes cluster.
  • The CIDR block of Services cannot overlap with the CIDR blocks of vSwitches.
  • The CIDR block of Services cannot overlap with the CIDR blocks of Pod vSwitches.
The CIDR block of Services. Service is an abstraction in Kubernetes. Each ClusterIP Service has an IP address. When you set this parameter, take note of the following items:
  • The IP address of a Service is effective only within the Kubernetes cluster.
  • The CIDR block of Services cannot overlap with the CIDR blocks of vSwitches.
  • The CIDR block of Services cannot overlap with the Pod CIDR block.
Service IPv6 CIDR If you enable IPv6 dual-stack, you must specify an IPv6 CIDR block for Services. The IPv6 CIDR block must range from fc00::/112 to fc00::/120. We recommend that you specify an IPv6 CIDR block that has the same number of IP addresses as the Service CIDR block. You do not need to set this parameter if you install Flannel.

Network Planning

To use Kubernetes clusters that are supported by ACK on Alibaba Cloud, you must first set up networks for the clusters based on the cluster sizes and business scenarios. You can refer to the following tables to set up networks for Kubernetes clusters. Change specifications as needed in unspecified scenarios.

Plan the network of a VPC

Cluster size Scenario VPC Zone
< 100 nodes Regular business Single VPC 1
Unlimited Cross-zone deployment Single VPC ≥ 2
Unlimited High reliability and cross-region deployment Multiple VPCs ≥ 2

Plan CIDR blocks for clusters

The following tables describe how to plan CIDR blocks for clusters that use Flannel or Terway.
  • Clusters that use Flannel
    VPC CIDR block vSwitch CIDR block Pod CIDR block Service CIDR block Maximum number of pod IP addresses
    192.168.0.0/16 192.168.0.0/24 172.20.0.0/16 172.21.0.0/20 65536
  • Clusters that use Terway
    • Table 1. Exclusive elastic network interface (ENI) mode or IPVLAN mode is enabled
      VPC CIDR block vSwitch CIDR block CIDR block of pod vSwitches Service CIDR block Maximum number of pod IP addresses
      192.168.0.0/16 192.168.0.0/19 192.168.32.0/19 172.21.0.0/20 8192
    • Table 2. Multi-zone deployment
      VPC CIDR block vSwitch CIDR block CIDR block of pod vSwitches Service CIDR block Maximum number of pod IP addresses
      192.168.0.0/16 Zone I 192.168.0.0/19 192.168.32.0/19 172.21.0.0/20 8192
      Zone J 192.168.64.0/19 192.168.96.0/19 8192

How to plan CIDR blocks

  • Scenario 1: One VPC and one Kubernetes cluster

    This is the simplest scenario. The CIDR block of a VPC is specified when you create the VPC. When you create a cluster in the VPC, make sure that the CIDR block of pods and the CIDR block of Services do not overlap with the VPC CIDR block.

  • Scenario 2: One VPC and multiple Kubernetes clusters
    You want to create more than one cluster in a VPC.
    • The CIDR block of the VPC is specified when you create the VPC. When you create clusters in the VPC, make sure that the VPC CIDR block, Service CIDR block, and pod CIDR block of each cluster do not overlap with one another.
    • The Service CIDR blocks of the clusters can overlap with each other. However, the pod CIDR blocks cannot overlap with each other.
    • In the default network mode (Flannel), the packets of pods must be forwarded by the VPC router. ACK automatically generates a route table for each destination pod CIDR block on the VPC router.
    Note In this case, a pod in one cluster can communicate with the pods and ECS instances in another cluster. However, the pod cannot communicate with the Services in another cluster.
  • Scenario 3: Two connected VPCs
    If two VPCs are connected, you can use the route table of one VPC to specify the packets that you want to send to the other VPC. The CIDR block of VPC 1 is 192.168.0.0/16 and the CIDR block of VPC 2 is 172.16.0.0/12, as shown in the following figure. You can use the route table of VPC 1 to forward all packets that are destined for 172.16.0.0/12 to VPC 2.
    Route Table
    Table 3. Connected VPCs
    VPC CIDR block Destination CIDR block Destination VPC
    VPC 1 192.168.0.0/16 172.16.0.0/12 VPC 2
    VPC 2 172.16.0.0/12 192.168.0.0/16 VPC 1
    In this scenario, make sure that the following conditions are met when you create a cluster in VPC 1 or VPC 2:
    • The CIDR blocks of the cluster do not overlap with the CIDR block of VPC 1.
    • The CIDR blocks of the cluster do not overlap with the CIDR block of VPC 2.
    • The CIDR blocks of the cluster do not overlap with those of other clusters in VPC 1 and VPC 2.
    • The CIDR blocks of the cluster do not overlap with those of pods in VPC 1 and VPC 2.
    • The CIDR blocks of the cluster do not overlap with those of Services in VPC 1 and VPC 2.

    In this example, you can set the pod CIDR block of the cluster to a subset of 10.0.0.0/8.

    Note All IP addresses in the destination CIDR block of VPC 2 can be considered in use. Therefore, the CIDR blocks of the cluster cannot overlap with the destination CIDR block.

    To access pods in VPC 1 from VPC 2, you must configure a route in VPC 2. The route must point to the pod CIDR block of a cluster in VPC 1.

  • Scenario 4: One VPC and one data center

    If a VPC is connected to a data center, packets of specific CIDR blocks are routed to the data center. In this case, the pod CIDR block of a cluster in the VPC cannot overlap with these CIDR blocks. To access pods in the VPC from the data center, you must configure a route in the data center to enable VBR-to-VPC peering connection.