Documentation
Home Page Container Service for Kubernetes ACK One User Guide Multi-cluster Fleets Multi-cluster gateways Get started Zone-disaster recovery system

Zone-disaster recovery system

Updated at:
Copy as MD
Product

You can use the ACK One ALB multi-cluster gateway with ACK One GitOps or multi-cluster application distribution to quickly implement active-active zone-disaster recovery for your applications. This ensures high availability and provides automatic failover. This topic describes how to build a zone-disaster recovery system by using an ALB multi-cluster gateway.

Disaster recovery overview

Cloud-based disaster recovery solutions are typically categorized into three types:

  • Cross-AZ zone-disaster recovery: This type includes both active-active and active-passive strategies. Because data centers within the same region are physically close, network latency is low. This setup protects against AZ-level disasters such as fires, network outages, or power failures. This solution is practical, offering simple data backup and fast recovery.

  • Active geo-redundancy: While this approach results in higher network latency, it protects against region-level disasters such as earthquakes and floods.

  • Two-region, three-center: This model combines a dual-center setup in one region with a disaster recovery site in another, offering the advantages of both. It is ideal for scenarios that demand high application and data continuity and availability.

From a business architecture perspective, a typical enterprise system is divided into an access layer, an application layer, and a data layer.

  • Access layer: Serves as the traffic entry point, receiving and forwarding traffic to the backend application layer based on routing rules.

  • Application layer: Contains the application services that process data based on requests and return responses to the upstream layer.

  • Data layer: Provides data storage services for the application layer.

To achieve end-to-end business disaster recovery, you must implement disaster recovery measures at each of these layers.

  • Access layer: The ACK One ALB multi-cluster gateway serves as the access layer and provides built-in high availability across AZs within the same region.

  • Application layer: The ACK One ALB multi-cluster gateway handles disaster recovery for the application layer, enabling active-active/active-passive zone-disaster recovery and active geo-redundancy.

  • Data layer: Disaster recovery and data synchronization at the data layer depend on the capabilities of the middleware being used.

Benefits

Using the ACK One ALB multi-cluster gateway for disaster recovery offers the following advantages over DNS-based solutions:

  • DNS-based disaster recovery requires multiple load balancer IP addresses (one for each cluster). In contrast, a gateway-based solution requires only one load balancer IP per region and provides high availability across multiple AZs by default.

  • The gateway-based solution supports Layer 7 routing, whereas DNS-based solutions do not.

  • In DNS-based solutions, IP address changes can cause temporary service disruptions due to client-side DNS caching. The gateway-based solution enables seamless traffic failover to the service backend in another cluster.

  • The multi-cluster gateway is a regional resource. All operations are managed from a central Fleet instance, eliminating the need to install an Ingress controller and create Ingress resources in each ACK cluster. This provides regional traffic management while reducing multi-cluster management overhead.

Solution architecture

This topic uses a sample web application, which includes Deployment and Service resources, to demonstrate the architecture of a zone-disaster recovery solution built with an ALB multi-cluster gateway.

image

  • Create two ACK clusters, Cluster 1 and Cluster 2, in two different AZs (AZ 1 and AZ 2) within the same region.

  • Use ACK One GitOps to distribute the application to Cluster 1 and Cluster 2.

  • Create an ALB multi-cluster gateway by creating an AlbConfig resource in the ACK One Fleet instance.

  • After creating the ALB multi-cluster gateway, create an Ingress to route traffic based on weights or headers. If one cluster becomes unavailable, traffic is automatically routed to the healthy cluster.

  • Data synchronization for ApsaraDB RDS depends on the capabilities of the middleware.

Prerequisites

Step 1: Deploy applications to multiple clusters

ACK One supports deploying applications to multiple clusters through multi-cluster GitOps or multi-cluster application distribution. For more information, see Quick start for GitOps, Create a multi-cluster application, and Quick start for application distribution. This topic uses GitOps as an example.

  1. Log on to the ACK One console. In the navigation pane on the left, choose Fleet > Multi-cluster GitOps.

  2. In the upper-left corner of the Multi-cluster GitOps page, click the Dingtalk_20231226104633.jpg button next to the fleet name, and select the target fleet from the drop-down list.

  3. Click Create Multi-cluster Application > GitOps to go to the Create Multi-cluster Application - GitOps page.

    Note

  4. On the Create from YAML tab, copy the following YAML content into the editor, and then click OK to create and deploy the application.

    Note

    The following YAML deploys the web-demo application to all associated clusters. You can also select specific clusters on the Quick Create tab, and your selections will be reflected in the content on the Create from YAML tab.

    apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: appset-web-demo
  namespace: argocd
spec:
  template:
    metadata:
      name: '{{.metadata.annotations.cluster_id}}-web-demo'
      namespace: argocd
    spec:
      destination:
        name: '{{.name}}'
        namespace: gateway-demo
      project: default
      source:
        repoURL: https://github.com/AliyunContainerService/gitops-demo.git
        path: manifests/helm/web-demo
        targetRevision: main
        helm:
          valueFiles:
            - values.yaml
          parameters:
            - name: envCluster
              value: '{{.metadata.annotations.cluster_name}}'
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true
  generators:
    - clusters:
        selector:
          matchExpressions:
            - values:
                - cluster
              key: argocd.argoproj.io/secret-type
              operator: In
            - values:
                - in-cluster
              key: name
              operator: NotIn
  goTemplateOptions:
    - missingkey=error
  syncPolicy:
    preserveResourcesOnDeletion: false
  goTemplate: true

Step 2: Create an ALB multi-cluster gateway

Create an AlbConfig object in the ACK One Fleet instance to create an ACK One ALB multi-cluster gateway and associate clusters with it.

  1. Obtain two vSwitch IDs from the VPC where the ACK One Fleet is located.

  2. Create a file named gateway.yaml with the following content.

    Note

    • Replace ${vsw-id1} and ${vsw-id2} with the vSwitch IDs from the previous step. Replace ${cluster1} and ${cluster2} with the IDs of the associated clusters that you want to add.

    • For the associated clusters ${cluster1} and ${cluster2}, configure the inbound rules of their security groups to allow traffic from the vSwitch CIDR block on all ports.

    apiVersion: alibabacloud.com/v1
kind: AlbConfig
metadata:
  name: ackone-gateway-demo
  annotations:
    # Add the associated clusters that will process traffic to the ALB multi-cluster instance.
    alb.ingress.kubernetes.io/remote-clusters: ${cluster1},${cluster2}
spec:
  config:
    name: one-alb-demo
    addressType: Internet
    addressAllocatedMode: Fixed
    zoneMappings:
    - vSwitchId: ${vsw-id1}
    - vSwitchId: ${vsw-id2}
  listeners:
  - port: 8001
    protocol: HTTP
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: alb
spec:
  controller: ingress.k8s.alibabacloud/alb
  parameters:
    apiGroup: alibabacloud.com
    kind: AlbConfig
    name: ackone-gateway-demo

    The following table describes the parameters.

    Parameter

    Required

    Description

    metadata.name

    Yes

    The name of the AlbConfig.

    metadata.annotations:

    alb.ingress.kubernetes.io/remote-clusters

    Yes

    The associated clusters to add to the ALB multi-cluster gateway. The cluster IDs specified here must already be associated with the Fleet instance.

    spec.config.name

    No

    The name of the ALB instance.

    spec.config.addressType

    No

    The network type of the ALB instance. Valid values:

    • Internet (default): An internet-facing instance that provides services over the internet.

      Note

      Application Load Balancer uses an elastic IP address (EIP) to provide services over the internet. If you use an internet-facing ALB instance, you are charged instance fees and bandwidth or data transfer fees for the EIP. For more information, see Pay-as-you-go.

      Intranet: A private network instance that provides services within a VPC and cannot be accessed from the internet.

    spec.config.zoneMappings

    Yes

    The IDs of the vSwitches for the ALB instance. For more information about how to create a vSwitch, see Create and manage vSwitches.

    Note

    • The specified vSwitches must be in availability zones supported by ALB and in the same VPC as your clusters. For more information about the regions and availability zones supported by ALB, see Regions and zones.

    • For high availability, select vSwitches from at least two different availability zones if the region supports more than one.

    spec.listeners

    No

    The listener port and protocol of the ALB instance. This example configures an HTTP listener on port 8001.

    A listener defines how traffic enters the load balancer. Retain this configuration; otherwise, you must create a listener before using the ALB Ingress.

  3. Run the following command to deploy gateway.yaml to create the ALB multi-cluster gateway and IngressClass:

    kubectl apply -f gateway.yaml

  4. Wait 1 to 3 minutes, then run the following command to verify that the ALB multi-cluster gateway is created:

    kubectl get albconfig ackone-gateway-demo

    Expected output:

    NAME      		      ALBID      DNSNAME                                  PORT&PROTOCOL   CERTID   AGE
ackone-gateway-demo           alb-xxxx   alb-xxxx.<regionid>.alb.aliyuncsslb.com                           4d9h

  5. Run the following command to verify that the associated clusters have been added:

    kubectl get albconfig ackone-gateway-demo -ojsonpath='{.status.loadBalancer.subClusters}'

    The expected output is a list of cluster IDs.

Step 3: Implement disaster recovery with Ingress

The multi-cluster gateway uses an Ingress to manage traffic across multiple clusters. Create an Ingress object in the ACK One Fleet instance to implement active-active zone-disaster recovery.

  1. In the Fleet instance, create the namespace where the Service resides. In this example, the namespace is gateway-demo.

  2. Create a file named ingress-demo.yaml with the following content.

    Note

    • The sum of the weights specified in the alb.ingress.kubernetes.io/cluster-weight annotations must be 100.

    • This routing rule exposes the backend service service1 at the path /svc1 on the domain alb.ingress.alibaba.com. Replace ${cluster1-id} and ${cluster2-id} with your cluster IDs.

    apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/listen-ports: |
     [{"HTTP": 8001}]
    alb.ingress.kubernetes.io/cluster-weight.${cluster1-id}: "20"
    alb.ingress.kubernetes.io/cluster-weight.${cluster2-id}: "80"
  name: web-demo
  namespace: gateway-demo
spec:
  ingressClassName: alb
  rules:
  - host: alb.ingress.alibaba.com
    http:
      paths:
      - path: /svc1
        pathType: Prefix
        backend:
          service:
            name: service1
            port:
              number: 80

  3. Run the following command to deploy the Ingress in the ACK One Fleet instance:

    kubectl apply -f ingress-demo.yaml -n gateway-demo

Step 4: Verify zone-disaster recovery

Verify traffic routing ratio

Access the service by using the following command:

curl -H "host: alb.ingress.alibaba.com" alb-xxxx.<regionid>.alb.aliyuncsslb.com:<listeners port>/svc1

The following table describes the parameters.

Parameter

Description

alb-xxxx.<regionid>.alb.aliyuncsslb.com

The DNSNAME of the AlbConfig that you obtained in Step 2.

<listeners port>

The listener port (8001) that is defined in the AlbConfig and declared in the Ingress annotations.

Run the following command. The result shows that requests are routed to Cluster 1 (poc-ack-1) and Cluster 2 (poc-ack-2) at a 20:80 ratio.

for i in {1..500}; do curl -H "host: alb.ingress.alibaba.com" alb-xxxx.cn-beijing.alb.aliyuncsslb.com:8001/svc1; done > res.txt
grep poc-ack-1 res.txt |wc -l
      108
grep poc-ack-2 res.txt |wc -l
      392

Verify seamless traffic failover

Run the following command. While the command is running, manually scale the number of application replicas in Cluster 2 to 0. Traffic will automatically fail over to Cluster 1.

for i in {1..500}; do curl -H "host: alb.ingress.alibaba.com" alb-xxxx.cn-beijing.alb.aliyuncsslb.com:8001/svc1; sleep 1; done
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-2!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
This is poc-ack-1!
Previous：Get startedNext：Build an active zone-redundancy system in a hybrid cloud environment