Alibaba Cloud Service Mesh (ASM) provides cross-region traffic distribution and failover
capabilities for applications. The cross-region traffic distribution feature implements
cross-region load balancing by routing traffic to multiple clusters based on their
weights. The cross-region failover feature implements cross-region disaster recovery
by transferring traffic from a faulty region to another region. This topic shows you
how to use the cross-region failover and traffic distribution features to implement
cross-region disaster recovery and load balancing. In the example of this topic, the
Bookinfo application is used.
Plan a network
Before you use ASM, you must complete network configurations for ASM. This involves
the CIDR blocks and names of vSwitches, virtual private clouds (VPCs), and clusters.
In this example, a network is created based on the following plan:
- Network plan for vSwitches and VPCs
- vSwitches
Notice To prevent route conflicts when you use Cloud Enterprise Network (CEN) to connect
to a VPC, specify a unique CIDR block for each vSwitch.
Category |
vSwitch |
VPC |
IPv4 CIDR block |
Cluster |
vpc-hangzhou-switch-1 |
vpc-hangzhou |
20.0.0.0/16 |
vpc-shanghai-switch-1 |
vpc-shanghai |
21.0.0.0/16 |
ASM instance |
vpc-hangzhou-switch-2 |
vpc-hangzhou2 |
192.168.0.0/24 |
- VPCs
Category |
VPC |
Region |
IPv4 CIDR block |
Cluster |
vpc-hangzhou |
cn-hangzhou |
20.0.0.0/8 |
vpc-shanghai |
cn-shanghai |
21.0.0.0/8 |
ASM instance |
vpc-hangzhou2 |
cn-hangzhou |
192.168.0.0/16 |
- Network plan for pods and services in clusters
Cluster |
Region |
VPC |
Pod CIDR block |
Service CIDR block |
ack-hangzhou |
cn-hangzhou |
vpc-hangzhou |
10.0.0.0/16 |
172.16.0.0/16 |
ack-shanghai |
cn-shanghai |
vpc-shanghai |
10.1.0.0/16 |
172.17.0.0/16 |
Step 1: Create clusters in different regions
- Create two vSwitches in the China (Hangzhou) and China (Shanghai) regions based on
the preceding plan, and then create VPCs that are associated with the vSwitches. For
more information, see Create a vSwitch and Create a VPC.
- Use the VPCs that you created and the preceding network plan to create clusters in
the China (Hangzhou) and China (Shanghai) regions. For more information, see Create an ACK managed cluster.
- Create an ASM instance in the China (Hangzhou) region based on the preceding network
plan. For more information, see Create an ASM instance.
Step 2: Use CEN to implement cross-region VPC communication
You can connect the VPCs among clusters and those between the clusters and the ASM
instance by using CEN.
- Create a CEN instance.
Note In this example, a CEN instance is created in the earlier version of the CEN console.
You can click Previous Version in the upper-right corner of the CEN console to use the earlier version.
- Log on to the CEN console.
- On the Instances page, click Create CEN Instance.
- In the Create CEN Instance panel, set the parameters and click OK.
Parameter |
Description |
Name |
The name of the CEN instance.
The name must be 2 to 128 characters in length and can contain digits, underscores
(_), and hyphens (-). The name must start with a letter.
|
Description |
The description of the CEN instance. |
Network Type |
The type of the network instance that you want to attach to the CEN instance. In this
example, VPC is used.
|
Region |
The region in which the network instance that you want to attach resides. In this
example, China (Hangzhou) is used.
|
Networks |
The network instance that you want to attach. In this example, the VPC that you created
in the China (Hangzhou) region is used.
|
- Attach network instances.
- Attach the VPC of the cluster that resides in the China (Shanghai) region.
- On the Instances page, find the CEN instance that you created and click its ID.
- Click the Networks tab, and then click Attach Network.
- In the Attach Network panel, set the parameters on the Your Account tab and click OK.
Parameter |
Description |
Network Type |
The type of the network instance that you want to attach to the CEN instance. In this
example, VPC is used.
|
Region |
The region in which the network instance that you want to attach resides. In this
example, China (Shanghai) is used.
|
Networks |
The network instance that you want to attach. In this example, the VPC that you created
in the China (Shanghai) region is used.
|
- Attach the VPC of the ASM instance that resides in the China (Hangzhou) region.
- Click the Networks tab, and then click Attach Network.
- In the Attach Network panel, set the parameters on the Your Account tab and click OK.
Parameter |
Description |
Network Type |
The type of the network instance that you want to attach to the CEN instance. In this
example, VPC is used.
|
Region |
The region in which the network instance that you want to attach resides. In this
example, China (Hangzhou) is used.
|
Networks |
The network instance that you want to attach. In this example, the VPC that you created
in the China (Hangzhou) region is used.
|
- Purchase a bandwidth plan. For more information, see Purchase a bandwidth plan.
- Set the cross-region connection bandwidth.
- On the Instances page, find the CEN instance that you created and click its ID.
- Click the Region Connections tab, and then click Set Region Connection.
- In the Set Region Connection panel, select the bandwidth plan that you purchased from the Bandwidth Plans drop-down list, set the Connected Regions parameter to China (Shanghai) and China (Hangzhou), and then click OK.
- Add security group rules.
Add the pod CIDR block of the ack-shanghai cluster to the security group of the ack-hangzhou
cluster and vice versa. This allows IP addresses from within the pod CIDR block of
a cluster to access the other cluster.
- Log on to the ACK console.
- On the Clusters page, find the ack-shanghai cluster and click Details in the Actions column.
- On the Cluster Information page, click the Basic Information tab.
View the pod CIDR block of the ack-shanghai cluster and go back to the Clusters page.
- On the Clusters page, find the ack-hangzhou cluster and click Details in the Actions column.
- On the Cluster Information page, click the Cluster Resources tab. Then, click the security group ID next to Security Group.
- On the Security Group Rules page, click Add Rule on the Inbound tab.
- Set the Protocol Type parameter to All and the Source parameter to the pod CIDR block of the ack-shanghai cluster. Then, click Save in the Actions column.
- Repeat the preceding substeps to view the pod CIDR block of the ack-hangzhou cluster
and add the pod CIDR block to the security group of the ack-shanghai cluster.
Step 3: Publish the pod route information to CEN
- Log on to the ACK console.
- On the Clusters page, find the ack-hangzhou cluster and click Details in the Actions column.
- On the cluster details page, click the Cluster Resources tab and click the VPC ID next to VPC.
- On the Information tab, view the router ID in the vRouter Basic Information section.
- In the left-side navigation pane of the VPC console, click Route Tables.
- On the Route Tables page, find the router ID and click the name of the route instance.
- On the Route Entry List tab, click Custom.
- Click Publish next to the pod CIDR block. In this example, a CIDR block of 10.45.0.0/16 is used.
- In the Publish Route Entry dialog box, click OK.
- Repeat the preceding substeps to publish the pod route information about the ack-shanghai
cluster to CEN.
- Verify whether the pod route information about the ack-hangzhou and ack-shanghai clusters
is published to CEN.
On the details page of the route table in the China (Hangzhou) region, click Dynamic on the Routes tab. You can view the route information of the pod CIDR block in the ack-shanghai
cluster and the route information about the IPv4 CIDR block of vpc-shanghai-switch-1.
On the details page of the route table in the China (Shanghai) region, you can also
view the route information of the pod CIDR block in the ack-hangzhou cluster and the
route information about the IPv4 CIDR block of vpc-hangzhou-switch-1. This indicates
that the pod route information about the ack-hangzhou and ack-shanghai clusters is
published to CEN.
Step 4: Add clusters to an ASM instance
Add the ack-hangzhou and ack-shanghai clusters that you created to an ASM instance.
- Log on to the ASM console.
- In the left-side navigation pane, choose .
- On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM
instance or click Manage in the Actions column.
- On the details page of the ASM instance, choose in the left-side navigation pane. On the Kubernetes Clusters page, click Add.
- In the Add Cluster panel, select the ack-hangzhou cluster and click OK.
- In the Note dialog box, click OK.
- Repeat the preceding substeps to add the ack-shanghai cluster to the same ASM instance.
Step 5: Configure an ingress gateway in ASM
- View the ID of the ack-shanghai cluster.
- Log on to the ACK console.
- On the Clusters page, find the ack-shanghai cluster and click Details in the Actions column.
- On the Cluster Information page, click the Basic Information tab.
In the Basic Information section, view the ID of the ack-shanghai cluster.
- Log on to the ASM console.
- In the left-side navigation pane, choose .
- On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM
instance or click Manage in the Actions column.
- On the details page of the ASM instance, click ASM Gateways in the left-side navigation pane. On the ASM Gateways page, click Create.
- On the Create page, select the ack-hangzhou cluster from the Cluster drop-down list, set the SLB Instance Type parameter to Internet Access, and then select a Server Load Balancer (SLB) instance from the Create SLB Instance drop-down list. Use the default values for other parameters. Then, click Create.
- On the ASM Gateways page, find a gateway named ingressgateway and click YAML in the Actions column.
- In the Edit panel, enter the ID of the ack-shanghai cluster and click OK.
spec:
clusterIds:
- ack-hangzhou cluster-id
- ack-shanghai cluster-id
Step 6: Deploy the Bookinfo application
- Use kubectl to connect to the ack-hangzhou cluster. For more information, see Connect to ACK clusters by using kubectl.
- Create an ack-hangzhou-k8s.yaml file that contains the following content:
- Run the following command to deploy the Bookinfo application in the ack-hangzhou cluster:
kubectl apply -f ack-hangzhou-k8s.yaml
- Use kubectl to connect to the ack-shanghai cluster. For more information, see Connect to ACK clusters by using kubectl.
Note When you use kubectl to connect to the ack-shanghai cluster, you must switch the kubeconfig
of the ack-hangzhou cluster to that of the ack-shanghai cluster.
- Create an ack-shanghai.yaml file that contains the following content:
- Run the following command to deploy the Bookinfo application in the ack-shanghai cluster:
kubectl apply -f ack-shanghai.yaml
- Use kubectl to connect to the ASM instance. For more information, see Use kubectl to connect to an ASM instance.
Note When you use kubectl to connect to the ASM instance, you must switch the kubeconfig
of the ack-shanghai cluster to that of the ASM instance.
- Create an asm.yaml file that contains the following content:
- Run the following command to create a routing rule in the ASM instance:
kubectl apply -f asm.yaml
- Verify whether the Bookinfo application is deployed.
- Log on to the ACK console.
- In the left-side navigation pane of the ACK console, click Clusters.
- On the Clusters page, find the ack-hangzhou cluster and click Details in the Actions column.
- In the left-side navigation pane of the details page, choose
- At the top of the Services page, select istio-system from the Namespace drop-down list. Find an ingress gateway named istio-ingressgateway and view the IP
address whose port is 80 in the External Endpoint column.
- Enter <IP address of the ingress gateway>/productpage in the address bar of your browser.
Refresh the page multiple times. The following images alternately appear on the screen.


Step 7: Use the cross-region traffic distribution and failover features
Configure cross-region traffic distribution
- Log on to the ASM console.
- In the left-side navigation pane, choose .
- On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM
instance or click Manage in the Actions column.
- On the details page of the ASM instance, choose in the left-side navigation pane. On the Basic Information page, click Settings.
- On the Basic Information page, click Enable locality traffic distribution on the right of Locality-Failover.
Note If you have enabled cross-region failover, you must disable cross-region failover
before you can enable cross-region traffic distribution.
- In the Locality-Traffic-Distribution dialog box, set the Policy parameter to cn-hangzhou and click New Policy.
- Click the
icon and then the
icon. Set the To parameter to cn-hangzhou and the Weight parameter to 90%.
- Click the
icon, set the To parameter to cn-shanghai and the Weight parameter to 10%, and then click Submit.
- Run the following command to request the Bookinfo application 10 times to verify whether
the cross-region traffic distribution is successful:
for ((i=1;i<=10;i++));do curl http://<Ingress gateway endpoint of port 80 in the ack-hangzhou cluster>/productpage 2>&1grep full.stars;done
Expected output:
<!-- full stars: -->
<!-- full stars: -->
You can find that 10 access requests are made and two rows of full stars
output are returned. This indicates that 9 of the 10 requests are routed to the v1
reviews service in the ack-hangzhou cluster and 1 request is routed to the v2 reviews
service in the ack-shanghai cluster. Traffic is routed to different clusters based
on the weights of the clusters.
Configure cross-region failover
- Disable the reviews service in the ack-hangzhou cluster.
- Log on to the ACK console.
- In the left-side navigation pane of the ACK console, click Clusters.
- In the left-side navigation pane of the details page, choose .
- On the Deployments page, set the Namespace parameter to default, find reviews-v1, and then click Scale in the Actions column.
- In the Scale dialog box, set the Desired Number of Pods parameter to 0 and click OK.
- Configure a destination rule.
Configure a destination rule. If the reviews service cannot be requested within 1
second, the reviews service will be ejected for 1 minute.
- Log on to the ASM console.
- In the left-side navigation pane, choose .
- On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM
instance or click Manage in the Actions column.
- On the details page of the ASM instance, choose in the left-side navigation pane.
- On the DestinationRule page, find the reviews service and click YAML in the Actions column.
- In the Edit panel, copy the following content to the code editor and click OK:
spec:
......
trafficPolicy:
connectionPool:
http:
maxRequestsPerConnection: 1
outlierDetection:
baseEjectionTime: 1m
consecutive5xxErrors: 1
interval: 1s
- maxRequestsPerConnection: specifies the maximum number of requests per connection.
- baseEjectionTime: specifies the minimum ejection duration.
- consecutive5xxErrors: specifies the number of consecutive errors.
- interval: specifies the time interval for ejection analysis.
- Enable cross-region failover.
- On the details page of the ASM instance, choose in the left-side navigation pane.
- On the Basic Information page, click Enable Locality-Failover on the right of Locality-Failover.
Note If you have enabled cross-region traffic distribution, you must disable cross-region
traffic distribution before you can enable cross-region failover.
- In the Locality-Failover dialog box, set the Failover parameter to cn-hangzhou if the From parameter is set to cn-shanghai. Set the Failover parameter to cn-shanghai if the
From parameter is set to cn-hangzhou. Then, click Submit.
- Run the following command to request the Bookinfo application 10 times and record
the number of successful routes to the v2 reviews service:
for ((i=1;i<=10;i++));do curl http://<Ingress gateway endpoint of port 80 in the ack-hangzhou cluster>/productpage 2>&1grep full.stars;donewc -l
Expected output:
20
You can find that 10 access requests are made and 20 rows of results are returned.
This is because a two-row result that contains full stars
is returned each time a route to the v2 reviews service succeeds. This indicates
that all 10 requests are routed to the v2 reviews service in the ack-shanghai cluster,
and the cross-region failover is successful.
FAQ
I connect the VPCs of Kubernetes clusters by using CEN. However, an error message
is displayed when I add the clusters to the ASM instance. What do I do?
If your clusters reside in different regions, you must purchase a cross-region data
transfer plan and configure valid settings for cross-region data transfer when you
connect the VPCs of the clusters by using CEN. Otherwise, the ASM instance fails to
connect to the clusters on the data plane. In this case, you fail to add the clusters
to the ASM instance.
To resolve this issue, you need to reconfigure valid settings for cross-region data
transfer in the CEN console to connect the VPCs of the clusters by using CEN. For
more information, see Step 2: Use CEN to implement cross-region VPC communication.