Container Service for Kubernetes:Overview of hybrid networks

Network mode of hybrid clusters

You can add ECS nodes to an external Kubernetes cluster that is registered with ACK to create a hybrid cluster and use the cluster to connect cloud and on-premises networks. To do this, you first need to set the network mode of the hybrid cluster. Design the network of the cluster nodes that are deployed in the data center based on your business requirements.

• If the size of the external Kubernetes cluster is medium and small (for example, less than 100 nodes) and the cluster does not require high network performance, you can select one of the following network modes:

  • Flannel VXLAN

  • Calico IPIP

  • Cilium VXLAN

• If the size of the external Kubernetes cluster is large or you want to create a large hybrid cluster and the cluster requires high network performance, you can select one of the following network modes:

  • Calico route reflection

  • Cilium BGP routing

In most cases, external Kubernetes clusters use a Calico routing mode. This topic provides an example on how to configure a Kubernetes cluster deployed in a data center to use a Calico routing mode. For container network plug-ins, we recommend that you choose a custom network plug-in provided by the cloud platform that you use. ACK provides the Terway plug-in to help you manage container networks. The following figure shows the networking of a hybrid cluster.

The private CIDR block of the data center is 192.168.0.0/24 and the CIDR block of the container network is 10.100.0.0/16. The on-premises network uses the Calico route reflection mode. The CIDR block of the virtual private cloud (VPC) is 10.0.0.0/8, the CIDR block of the vSwitch for compute nodes is 10.10.24.0/24, and the CIDR block of the vSwitch for pods is 10.10.25.0/24. The cloud network uses the One ENI for Multi-Pod mode of Terway.

The key to building a hybrid network is to connect the cloud network to the on-premises network. To do this, perform the following operations:

• Connect the on-premises network to the VPC.

• Connect the on-premises container network to the container network in the cloud.

Connect the cloud network to the on-premises network

To connect a cloud network to an on-premises network in cloud-native scenarios, you need to connect compute nodes and pods that are deployed in both networks. The following figure shows how nodes and pods are connected.

To connect the cloud network to the on-premises network, perform the following steps. For more information, see Connect a data center to ECS by using an Express Connect circuit.

1. Use an Express Connect circuit to connect the on-premises network to Alibaba Cloud.

   For more information about the corresponding solution, see Physical Connection.

2. Create a connection over an Express Connect circuit to connect edge devices in the data center to a virtual border router (VBR) that functions as a gateway in the cloud.

3. Attach the VBR and VPC to a Cloud Enterprise Network (CEN) instance.

4. Configure BGP on the VBR and in the data center.

5. Test the network connectivity between the cloud network and on-premises network.

6. Configure routes that point to the private CIDR blocks used by the cloud services to communicate with the on-premises network. For more information about the operations, see the following topics:

   • ACR: Add routes that point to the private addresses of ACK component images.

   • Prometheus: VPC endpoints and the corresponding CIDR blocks of Managed Service for Prometheus.

   • OSS: Internal endpoints of OSS buckets and VIP ranges.