A hybrid cluster connects a self-managed on-premises Kubernetes cluster to ACK through a registered cluster. It lets you scale cloud ECS nodes for the on-premises cluster and manage compute resources across cloud and on-premises environments. This topic describes hybrid network modes and how to interconnect cloud and on-premises networks.
Choose a network mode for your hybrid cluster and connect cloud and on-premises networks with Express Connect.
Prerequisites
Make sure that:
-
On-premises network meets the connectivity requirements for a hybrid cluster.
-
An external Kubernetes cluster is deployed in your data center and registered with ACK.
-
Alibaba Cloud account has permissions to create Express Connect circuits, virtual border routers (VBRs), and Cloud Enterprise Network (CEN) instances.
Choose a network mode
Choose based on cluster size and network performance requirements.
| Network mode | Networking model | When to use |
|---|---|---|
| Flannel VXLAN | Overlay | Clusters with fewer than 100 nodes; no high network performance requirement |
| Calico IPIP | Overlay | Clusters with fewer than 100 nodes; no high network performance requirement |
| Cilium VXLAN | Overlay | Clusters with fewer than 100 nodes; no high network performance requirement |
| Calico route reflection | BGP routing | Large clusters or clusters requiring high network performance |
| Cilium Border Gateway Protocol (BGP) routing | BGP routing | Large clusters or clusters requiring high network performance |
Most external Kubernetes clusters use Calico. This example uses Calico route reflection. ACK provides the Terway plug-in for cloud container networking.
Calico runs only on-premises. Terway runs only in the cloud. See Deploy and configure Terway.
How it works
The following figure shows the hybrid cluster network topology.
In this example configuration:
-
On-premises network uses Calico route reflection mode
-
Cloud network uses the One ENI for Multi-Pod mode of Terway
CIDR blocks used in this example:
| Network | CIDR |
|---|---|
| Data center private CIDR | 192.168.0.0/24 |
| Container network CIDR | 10.100.0.0/16 |
| Virtual private cloud (VPC) CIDR | 10.0.0.0/8 |
| vSwitch for compute nodes | 10.10.24.0/24 |
| vSwitch for pods | 10.10.25.0/24 |
Connect the cloud network to the on-premises network
Use Express Connect to establish a dedicated private connection between your data center and Alibaba Cloud.
See Connect a data center to ECS by using an Express Connect circuit.
-
Connect the on-premises network to Alibaba Cloud with an Express Connect circuit. See Physical Connection.
-
Create a connection over the Express Connect circuit to link data center edge devices to a VBR on the cloud side.
-
Attach the VBR and VPC to a CEN instance.
-
Configure BGP on the VBR and in the data center. See Configure BGP networks for data centers.
-
Test connectivity between the cloud and on-premises networks.
-
Configure routes to the private CIDR blocks used by cloud services for on-premises communication:
-
Container Registry: Add routes that point to the private address of ACK component images
-
Object Storage Service (OSS): Internal endpoints of OSS buckets and VIP ranges
-
Next steps
After connecting the networks, deploy and configure Terway on the cloud nodes to complete the hybrid cluster network setup: