When you connect a data center (IDC) to an Alibaba Cloud Virtual Private Cloud (VPC) using redundant Express Connect circuits, you must configure health checks on both the data center and Alibaba Cloud to monitor circuit connectivity.
Background
By default, Alibaba Cloud sends a probe packet from each health check source IP address to the destination IP address in your data center (IDC) every 2 seconds. If the probe packet returns over the probed Express Connect circuit, the circuit is considered healthy. If no response is received for eight consecutive probe packets over an Express Connect circuit, the circuit is considered faulty.
The destination IP address must be able to respond to probe packets. Do not rate-limit or block these probe packets.
Scenario | Routing method | Description |
Connect a data center (IDC) to Alibaba Cloud by using a VBR-to-VPC connection | static routing | You must add a route entry on the Virtual Border Router (VBR). The destination CIDR block must be the health check source IP address with a 32-bit subnet mask, and the next hop must point to the VPC. You must also manually configure a route entry in your data center (IDC). The destination CIDR block must be the health check source IP address with a 32-bit subnet mask, and the next hop must point to the corresponding Express Connect circuit. Otherwise, probe packets cannot return along the original path, which would cause Alibaba Cloud to incorrectly determine that the circuit is unavailable. |
dynamic routing | You must add a route entry on the VBR. The destination CIDR block must be the health check source IP address with a 32-bit subnet mask, and the next hop must point to the VPC. After the route entry is added, you must advertise this route from the VBR. | |
Connect a data center (IDC) to Alibaba Cloud by using Cloud Enterprise Network (CEN) | static routing | You must configure a health check between the source and destination IP addresses in the CEN console. You must also manually configure a route entry in your data center (IDC). The destination CIDR block must be the health check source IP address with a 32-bit subnet mask, and the next hop must point to the corresponding Express Connect circuit. |
dynamic routing | You must configure a health check between the source and destination IP addresses in the CEN console. After the health check is configured, Alibaba Cloud by default advertises the health check source IP address to your data center (IDC) as a /32 route. |
If policies such as Control Plane Policing (CoPP) (on Cisco devices) or other local anti-attack policies are configured on your gateway devices in your data center (IDC), probe packets may be dropped. This can cause health check flapping. We recommend that you disable such rate-limiting policies on the gateway devices.
Prerequisites
You have set up redundant connections:
Configure health checks for VBR-to-VPC
The VBR-to-VPC connection feature is not enabled by default. To use this feature, contact your account manager.
Static routing
Step 1: Configure VBR health check
If you use a cross-account VBR-to-VPC connection, you must use the acceptor account to configure the health check on the VBR.
Log on to the Express Connect console.
In the top navigation bar, select the region where your VBR is deployed. In the left-side navigation pane, choose .
On the VBR-to-VPC page, find the target peering connection and click Health Check in the Actions column.
In the Health Check panel, click Configure.
In the Edit VBR panel, configure the health check parameters and click OK.
Parameter
Description
Network Type
The network type of the VBR. This topic supports only IPv4 Routing.
Source IP
Enter an available private IP address from the interconnected VPC.
Destination IP
Enter the interface IP address of the gateway device in your data center (IDC).
Probe Interval (Seconds)
The interval at which probe packets are sent.
Default value: 2. Valid values: 2 to 3.
Failure Threshold
The number of consecutive probe packets that can be lost before the connection is declared unhealthy.
Default value: 8. Valid values: 3 to 8.
Step 2: Configure data center health check
This ensures that traffic fails over correctly between your redundant Express Connect circuits.
Before you configure health checks in your data center, you must configure return routes for the probe packets to ensure that the probe packets can return.
Configure return routes for probe packets in your data center.
Configuration commands vary by device vendor. The following example is for reference only. For the specific commands, consult your device vendor.
# Configure the return route for probe packets. ip route <Health Check Source IP 1> 255.255.255.255 <Alibaba Cloud-side IPv4 interconnect IP 1> ip route <Health Check Source IP 2> 255.255.255.255 <Alibaba Cloud-side IPv4 interconnect IP 2>This command adds a route entry on your data center's border router. The destination is the health check source IP address, with the next hop set to the Alibaba Cloud-side IPv4 interconnect IP of the VBR. This ensures probe packets are returned to the VPC over the correct circuit.
Implement a health check in your data center.
You can use Bidirectional Forwarding Detection (BFD) or Network Quality Analyzer (NQA) to check connectivity from your data center to the VBR. For specific commands, consult your device vendor.
ImportantWhen you configure an NQA probe, do not use the Alibaba Cloud-side IPv4 interconnect IP of the VBR as the probe destination. Otherwise, incorrect failovers may occur when the circuit is healthy, or failovers may not occur when the circuit is down. You must use the health check source IP address from Step 1 as the destination for probing the cloud network from your data center. This IP address supports only ICMP probes.
If you do not have a redundant Express Connect circuit, we recommend configuring a static summary route to the cloud that is independent of NQA results. This ensures that traffic continues to be forwarded even if an NQA probe fails while the circuit itself remains active.
Associate health checks with routing.
If your data center is connected to Alibaba Cloud through multiple Express Connect circuits, you must configure health check-based route association in your data center. This ensures that your data center can also detect the connectivity of the Express Connect circuits and automatically switch routes based on the health check results. For specific commands, consult your device vendor.
BGP routing
Step 1: Configure VBR health check
If you use a cross-account VBR-to-VPC connection, you must use the acceptor account to configure the health check on the VBR.
Log on to the Express Connect console.
In the top navigation bar, select the region where your VBR is deployed. In the left-side navigation pane, choose .
On the VBR-to-VPC page, find the target peering connection and click Health Check in the Actions column.
In the Health Check panel, click Configure.
In the Edit VBR panel, configure the health check parameters and click OK.
Parameter
Description
Network Type
The network type of the VBR. This topic supports only IPv4 Routing.
Source IP
Enter an available private IP address from the interconnected VPC.
Destination IP
Enter the interface IP address of the gateway device in your data center (IDC).
Probe Interval (Seconds)
The interval at which probe packets are sent.
Default value: 2. Valid values: 2 to 3.
Failure Threshold
The number of consecutive probe packets that can be lost before the connection is declared unhealthy.
Default value: 8. Valid values: 3 to 8.
Step 2: Add a VBR route to the VPC
Adding a route entry on the VBR that points to the VPC ensures that probe packets are correctly routed from the VBR to the VPC.
In the top navigation bar, select the region where your VBR is deployed. In the left-side navigation pane, click Virtual Border Routers (VBRs).
On the Virtual Border Routers (VBRs) page, click the ID of the target VBR instance.
On the VBR details page, click the tab, and then click Add Route.
In the Add Route panel, configure the parameters and click OK.
Parameter
Description
Next Hop Type
Select the type of the next hop.
In this topic, select VPC.
Destination CIDR Block
Enter the destination CIDR block.
In this topic, enter the health check source IP address with a /32 subnet mask. Example: 192.168.0.1/32.
Next Hop
Select the instance to use as the next hop.
In this topic, select the destination VPC instance.
Description
Enter a description for the route entry.
Step 3: Advertise the health check route
Before you advertise the route, ensure that a route entry pointing to the VPC is configured on the target VBR.
In the top navigation bar, select the region where your VBR is deployed. In the left-side navigation pane, click Virtual Border Routers (VBRs).
On the Virtual Border Routers (VBRs) page, click the ID of the target VBR instance.
On the VBR details page, click the Advertise BGP Subnet tab, and then click Advertise BGP Subnet.
In the Advertise BGP Subnet panel, enter the health check source IP address with a /32 subnet mask in the Advertised Subnet text box (for example, 192.168.0.1/32), and then click OK.
Configure health checks for CEN
Step 1: Configure the CEN health check
登录云企业网管理控制台。
In the left-side navigation pane, click Health Checks.
On the Health Checks page, select the region in which a VBR is deployed. Then, click Set Health Check.
In the Add Health Check panel, configure the following parameters and click OK.
Parameter
Description
Instances
The CEN instance to which the VBR is attached.
Virtual Border Router (VBR)
The VBR that you want to monitor.
Source IP
The source IP address. You can select one of the following methods to specify the source IP address:
Automatic IP Address: The system automatically assigns an IP address from the 100.96.0.0/16 CIDR block. We recommend that you select this option.
NoteIf you select this option and an ACL policy is configured on the peer , you must modify the ACL policy to allow this CIDR block. Otherwise, the health check fails.
Custom IP Address: You need to specify an idle IP address within the 10.0.0.0/8, 192.168.0.0/16, or 172.16.0.0/12 CIDR block. The specified IP address cannot be the IP address with which you want to communicate, the IP address of the VBR on the Alibaba Cloud side, or the IP address of the VBR on the user side.
Destination IP
The IP address of the VBR on the user side.
Probe Interval (Seconds)
The interval at which probe packets are sent for the health check. Unit: seconds.
Probe Packets
The number of probe packets that are sent for health checks. Unit: packet.
Change Route
Specifies whether to allow the health check feature to switch to the redundant route.
By default, Change Route is turned on. This indicates that the health check feature can switch to the redundant route. If a redundant route is configured on the CEN instance, the health check feature immediately switches to the redundant route if an error is detected on the Express Connect circuit.
If you turn off Change Route, the health check feature does not switch to the redundant route. Only probing is performed. The health check feature does not switch to the redundant route even if an error is detected on the Express Connect circuit.
WarningBefore you turn off Change Route, make sure that the system can switch to a redundant route by using other mechanisms. Otherwise, network connections are interrupted if the Express Connect circuit is down.
Step 2: Configure data center health check
You must add the required health check configurations in your data center to ensure the feature works correctly.
Add a return route for probe packets in your data center.
Important-
If your VBR instance uses Border Gateway Protocol (BGP), Alibaba Cloud automatically advertises the health check source IP address (with a 32-bit mask) to your data center after you configure the health check. You do not need to configure a return route for the probe packets in your data center.
-
If your VBR instance uses static routing, you must manually configure a route entry in your data center. The destination of the route entry must be the health check source IP address with a 32-bit subnet mask, and the next hop must point to the Alibaba Cloud side of the corresponding Express Connect circuit. Otherwise, the health check ping packets cannot return along the original path, and Alibaba Cloud incorrectly determines that the Express Connect circuit is unavailable.
Configuration commands vary by device vendor. The following example is for reference only. For the specific commands, consult your device vendor.
# Configure the return route for probe packets. ip route <Health Check Source IP 1> 255.255.255.255 <Alibaba Cloud-side IPv4 interconnect IP 1> ip route <Health Check Source IP 2> 255.255.255.255 <Alibaba Cloud-side IPv4 interconnect IP 2>This command adds a route entry on your data center's border router. The destination is the health check source IP address, with the next hop set to the Alibaba Cloud-side IPv4 interconnect IP of the VBR. This ensures probe packets are returned to the VPC over the correct circuit.
-
Implement a health check in your data center.
You can use Bidirectional Forwarding Detection (BFD) or Network Quality Analyzer (NQA) to check connectivity from your data center to the VBR. For specific commands, consult your device vendor.
ImportantWhen you configure an NQA probe, do not use the Alibaba Cloud-side IPv4 interconnect IP of the VBR as the probe destination. Otherwise, incorrect failovers may occur when the circuit is healthy, or failovers may not occur when the circuit is down. You must use the health check source IP address from Step 1 as the destination for probing the cloud network from your data center. This IP address supports only ICMP probes.
If you do not have a redundant Express Connect circuit, we recommend configuring a static summary route to the cloud that is independent of NQA results. This ensures that traffic continues to be forwarded even if an NQA probe fails while the circuit itself remains active.
Associate health checks with routing.
If your data center is connected to Alibaba Cloud through multiple Express Connect circuits, you must configure health check-based route association in your data center. This ensures that your data center can also detect the connectivity of the Express Connect circuits and automatically switch routes based on the health check results. For specific commands, consult your device vendor.
More operations
The following describes additional operations for health checks on the VBR side of Express Connect. For more information about health checks on the CEN side, see Configure a health check for a VBR.
Clear a health check
You can clear the health check configuration from a VBR.
Log on to the Express Connect console.
In the top navigation bar, select the region. In the left-side navigation pane, choose .
On the VBR-to-VPC page, find the target peering connection. In the Actions column, choose .
In the Health Check panel, click Clear. In the Clear Health Check Settings dialog box, click OK.
Configure CloudMonitor alerts for health checks
Log on to the CloudMonitor console.
-
In the left-side navigation pane, choose .
On the Alert Rules page, click Create Alert Rule.
In the Create Alert Rule panel, set Product to Express Connect - Peering Connections, Express Connect - VBR, or Express Connect - Physical Connections, configure the alert rule, and then click OK.
This section describes only the parameters that are most relevant to this topic. For information about other parameters, see Create an alert rule.
Click . In the Add Rule Description panel, configure the following parameters and click OK.
Parameter
Description
Rule Name
Enter a name for the alert rule.
Metric Type
The metric type for the alert rule. This topic uses Simple Metric as an example. For information about multi-metric and dynamic threshold configurations, see Create an alert template.
Monitoring indicators
Select the metric to monitor. The available health-check-related metrics are as follows:
Express Connect - Peering Connections
Healthy Check Loss Rate: The packet loss rate of health checks initiated from the VPC.
RouterInterfaceResponseTime: The latency of health checks initiated from the VPC. Unit: ms.
Express Connect - VBR
VbrHealthyCheckLatency: The latency of VBR health checks. Unit: μs.
VbrHealthyCheckLossRate: The packet loss rate of VBR health checks.
Express Connect - Physical Connections
PhysicalConnectionStatus: The connection status of the Express Connect circuit.
Threshold and Alert Level
Specify the conditions that trigger the alert, the threshold value, and the alert's severity level.
Chart Preview
A preview of the monitoring chart for the selected metric.
> Health Check