Container Service for Kubernetes:Disaster recovery scenario for multiple ACK clusters in different VPCs (based on CEN for VPC network connectivity)

How it works

ASM reads the region of each workload from cloud node labels and uses that information to make locality-aware routing decisions.

Both capabilities are configured through Geolocation-based Load Balancing on the ASM instance's Base Information page. Failover and traffic distribution are mutually exclusive — you must disable one before enabling the other.

Prerequisites

Before you begin, ensure that you have:

• An Alibaba Cloud account with permissions to create ACK clusters, ASM instances, VPCs, and Cloud Enterprise Network (CEN) instances

• Familiarity with kubectl and kubeconfig management for multiple clusters

Network configuration

Plan the following CIDR blocks before creating any resources. All vSwitches in the same CEN must use non-overlapping CIDR blocks to avoid route conflicts.

For CIDR planning guidance, see Plan CIDR blocks for multiple clusters on the data plane.

Step 1: Create clusters in different regions

1. Create the VPCs and vSwitches in the Hangzhou and Shanghai regions using the values in the network configuration tables above. See Create a vSwitch and Create a VPC and a vSwitch.

2. Create an ACK managed cluster in each region using the VPCs you created. See Create an ACK managed cluster.

3. Create an ASM instance in the Hangzhou region using the vpc-hangzhou2 VPC. See Create an ASM instance.

Step 2: Connect VPC networks across regions using CEN

Connect the cluster VPCs to each other and to the Service Mesh VPC using CEN transit routers.

1. Log on to the Cloud Enterprise Network (CEN) console and create a CEN instance. See Create a CEN instance.

2. Create two transit routers — one in each region: Create the following transit routers:

   1. On the CEN Instances page, click the name of your CEN instance. Under the Basic Information tab, click Create Transit Router.

   2. Set the Region and Name, then click OK.

   Name	Region
   hangzhou-router	China (Hangzhou)
   shanghai-router	China (Shanghai)

3. Add VPCs to each transit router. Repeat the following steps for each VPC:

   1. Click the transit router ID.

   2. On the Intra-region Connections tab, click Create Connection.

   3. Set Instance Type to Virtual Private Cloud (VPC) and select the VPC for the transit router's region.

   4. Keep all other settings as default and click OK.

4. Configure bandwidth for inter-region communication. After creation, the connection appears on the Inter-region Connections tab.

   1. Click the name of a transit router, then click Create Connection.

   2. In the Connection With Peer Network Instance dialog, set Region to the local region and Peer Region to the remote region.

   3. Configure the bandwidth and other parameters, then click OK. For details, see Inter-region connections.

   

5. Add inbound security group rules to allow cross-cluster traffic:

   1. Log on to the ACK console. In the left navigation pane, click Clusters.

   2. Get the Pod CIDR for each cluster:

      • Select the Shanghai region, click ack-shanghai, then on the Cluster Information page, click the Basic Information tab to find the Pod CIDR.

      • Repeat for ack-hangzhou in the Hangzhou region.

   3. Add an inbound security group rule to each cluster:

      1. On the Cluster Information page, click Basic Information, then click the security group ID next to Control Plane Security Group.

      2. On the Inbound tab, click Add Rule.

      3. Set Protocol Type to All and Source to the Pod CIDR of the peer cluster. Keep all other settings as default and click Save.

   4. Log on to a node in each cluster and run ping to confirm connectivity to the other cluster. See Log on to nodes.

   For the Flannel network plugin, add the Pod CIDR of the peer cluster as the source. For the Terway network plugin, add the vSwitch CIDR of the peer cluster instead. To find the vSwitch CIDR, log on to the VPC console and check the IPv4 CIDR Block column on the vSwitch page.

Step 3: Add clusters to ASM and create an ingress gateway

1. Add both the ack-hangzhou and ack-shanghai clusters to your ASM instance. See Add a cluster to an ASM instance.

2. Create a hosted ingress gateway by applying the following manifest:

   apiVersion: istio.alibabacloud.com/v1beta1
   kind: IstioGateway
   metadata:
     annotations:
       asm.alibabacloud.com/managed-by-asm: 'true'
     name: ingressgateway
     namespace: istio-system
   spec:
     gatewayType: ingress
     dnsPolicy: ClusterFirst
     externalTrafficPolicy: Local
     hostNetwork: false
     ports:
     - name: http
       port: 80
       protocol: TCP
       targetPort: 80
     - name: https
       port: 443
       protocol: TCP
       targetPort: 443
     replicaCount: 1
     resources:
       limits:
         cpu: '2'
         memory: 2G
       requests:
         cpu: 200m
         memory: 256Mi
     rollingMaxSurge: 100%
     rollingMaxUnavailable: 25%
     runAsRoot: true
     serviceType: LoadBalancer

Step 4: Deploy the Bookinfo application

1. Deploy the Bookinfo application in both clusters:

   kubectl apply -f bookinfo.yaml

2. Connect kubectl to the ASM instance and apply routing rules. Create a file named asm.yaml with the following content:

   View the complete YAML

   apiVersion: networking.istio.io/v1alpha3
   kind: Gateway
   metadata:
     name: bookinfo-gateway
   spec:
     selector:
       istio: ingressgateway # use istio default controller
     servers:
     - port:
         number: 80
         name: http
         protocol: HTTP
       hosts:
       - "*"
   ---
   apiVersion: networking.istio.io/v1alpha3
   kind: VirtualService
   metadata:
     name: bookinfo
   spec:
     hosts:
     - "*"
     gateways:
     - bookinfo-gateway
     http:
     - match:
       - uri:
           exact: /productpage
       - uri:
           prefix: /static
       - uri:
           exact: /login
       - uri:
           exact: /logout
       - uri:
           prefix: /api/v1/products
       route:
       - destination:
           host: productpage
           port:
             number: 9080
   ---
   apiVersion: networking.istio.io/v1alpha3
   kind: DestinationRule
   metadata:
     name: productpage
   spec:
     host: productpage
     subsets:
     - name: v1
       labels:
         version: v1
   ---
   apiVersion: networking.istio.io/v1alpha3
   kind: DestinationRule
   metadata:
     name: reviews
   spec:
     host: reviews
     subsets:
     - name: v1
       labels:
         version: v1
     - name: v2
       labels:
         version: v2
     - name: v3
       labels:
         version: v3
   ---
   apiVersion: networking.istio.io/v1alpha3
   kind: DestinationRule
   metadata:
     name: ratings
   spec:
     host: ratings
     subsets:
     - name: v1
       labels:
         version: v1
     - name: v2
       labels:
         version: v2
     - name: v2-mysql
       labels:
         version: v2-mysql
     - name: v2-mysql-vm
       labels:
         version: v2-mysql-vm
   ---
   apiVersion: networking.istio.io/v1alpha3
   kind: DestinationRule
   metadata:
     name: details
   spec:
     host: details
     subsets:
     - name: v1
       labels:
         version: v1
     - name: v2
       labels:
         version: v2

   Apply the routing rules:

   kubectl apply -f asm.yaml

3. Verify the deployment:

   1. Get the ingress gateway IP address.

   2. Open http://<ingress-gateway-IP>/productpage in a browser and refresh the page 10 times. The Bookinfo application distributes requests across the v1, v2, and v3 versions of the reviews service. After 10 refreshes, the traffic ratio across the three versions should be close to 1:1:1.

Step 5: Configure inter-region failover and traffic distribution

for ((i=1;i<=10;i++)); do
  curl http://<Port 80 of the IP address of the ingress gateway in the ack-hangzhou cluster>/productpage 2>&1 | grep full.stars
done | wc -l

20

for ((i=1;i<=10;i++)); do
  curl http://<Port 80 of the IP address of the ingress gateway in the ack-hangzhou cluster>/productpage 2>&1 | grep full.stars
done

<!-- full stars: -->
<!-- full stars: -->

FAQ

What's next

• Configure locality load balancing for services within a single region

• ASM instance management