When Transit Routers (TRs) from multiple regions in a Cloud Enterprise Network (CEN) are connected to an Express Connect Router (ECR), multiple traffic paths are available for communication between cloud and on-premises resources. This topic describes the path selection rules for this scenario.
Routing rules
The path for outbound traffic from the cloud is determined first. By default, when outbound traffic from a VPC reaches a TR:
If the current TR is directly connected to the ECR, the traffic is routed from the current TR to the ECR.
If the current TR is not directly connected to the ECR, the traffic is routed to another TR that is directly connected to the ECR. If multiple such TRs exist:
The TR in the same region as the destination VBR is selected.
If none of the other TRs are in the same region as the destination VBR, the TR in the region with the lexicographically smallest region ID is selected.
After the outbound traffic path is determined, the system ensures that the inbound traffic path is symmetric to the outbound path.
Scenario examples
This section provides two typical examples to help you understand the path selection rules.
Example one: Multiple TRs are connected to the ECR
In the scenario shown in the preceding figure, a company has a data center in the China (Hangzhou) region and has deployed services in VPCs in the China (Hangzhou) and China (Shanghai) regions.
The company uses a VBR, an ECR, and TRs to connect the on-premises data center to the cloud VPCs. A cross-region connection is established between the TR in Hangzhou and the TR in Shanghai. The ECR is attached to TR1 in Hangzhou, TR2 in Shanghai, and the VBR in Hangzhou.
1. Outbound traffic path
When VPC1 accesses the on-premises data center, traffic from VPC1 reaches TR1. Because TR1 is directly connected to the ECR, the traffic is routed from TR1 to the ECR.
When VPC2 accesses the on-premises data center, traffic from VPC2 reaches TR2. Because TR2 is also directly connected to the ECR, the traffic is routed from TR2 to the ECR.
2. Traffic path to the cloud
Because the outbound traffic paths are determined, the system directs inbound traffic to follow symmetric paths:
When the data center accesses VPC1, traffic reaches the ECR and is then routed through TR1 to VPC1.
When the data center accesses VPC2, traffic reaches the ECR and is then routed through TR2 to VPC2.
Example two: Multiple TRs are interconnected, and some are not connected to the ECR
In the scenario shown in the preceding figure, a company has connected three data centers to VBRs in the China (Beijing), China (Shanghai), and China (Hangzhou) regions. The company has also deployed services in VPCs in these three regions.
The company has connected the three VBRs to an ECR and has connected TR1 and TR2 to the ECR. Cross-region connections are established among all three TRs.
1. Outbound traffic path
VPC1 Egress Traffic Path | Outbound traffic path from VPC2 | VPC 3: Egress Traffic Path |
 |  |  |
When VPC1 sends outbound traffic to access the three data centers, the traffic is routed directly from TR1 to the ECR because TR1 is directly connected to the ECR.
The path for outbound traffic from VPC2 is similar to that from VPC1. The traffic is routed directly from TR2 to the ECR.
When VPC3 sends outbound traffic to access the three data centers, the traffic is routed to either TR1 or TR2 because TR3 is not directly connected to the ECR:
When VPC3 accesses IDC1, traffic reaches TR3 and is then routed to TR1, which is in the same region as the destination VBR1.
When VPC3 accesses IDC2, traffic reaches TR3 and is then routed to TR2, which is in the same region as the destination VBR2.
When VPC3 accesses IDC3, traffic reaches TR3. Because neither TR1 nor TR2 is in the same region as the destination VBR3, the traffic is routed to TR1, which has the lexicographically smaller region ID. The region ID of TR1 (cn-beijing) is lexicographically smaller than the region ID of TR2 (cn-shanghai).
2. Cloud-bound traffic path
Inbound traffic path to VPC1 | Inbound traffic path to VPC2 | Inbound traffic path to VPC3 |
 |  |  |
After the outbound traffic paths are determined, the system directs inbound traffic to follow symmetric paths. For more information, see the preceding figure. Details are not repeated here.
Change the default path selection behavior
In cases where a TR is selected by comparing the lexicographical order of region IDs, you can manually specify a TR to change the default traffic path. By changing the traffic path, you can:
Reduce latency: Change paths that might involve geographical detours to reduce network latency.
Lower costs: The bandwidth costs of different cross-region connection links in the cloud network may vary. You can prioritize low-cost cross-region links for your service traffic to reduce expenses.
Control traffic distribution: If a TR in a specific region experiences high traffic, you can direct some service traffic to a less busy TR to improve overall network quality and throughput.
Default traffic path between VPC3 and IDC3 | Traffic path between VPC3 and IDC3 after the change |
 |  |
As shown in the preceding figure, when VPC3 accesses IDC3, the path selection rules require a comparison of the lexicographical order of the region IDs for TR1 and TR2. By default, traffic is transmitted through TR1.
You can set TR2 as the default path to transmit traffic between VPC3 and IDC3 through TR2, thereby changing the default traffic path.
Procedure
Go to the Basic Information page of the target ECR. Find the target TR and click Set as Default in the Actions column.
How it works
The path selection rules between a TR and an ECR are implemented based on the route priority in their respective route tables. The following section explains the implementation details for users who require this information.
Click to view the implementation details of the path selection rules
MED description
The Multi-Exit Discriminator (MED) is a path attribute of the Border Gateway Protocol (BGP). When multiple paths for the same route exist between contiguous autonomous systems, you can set the MED attribute to change the traffic path.
Location of the MED attribute: The MED attribute is stored in a route entry. You can view it in the MED column of the ECR route entry list.
Value range of the MED attribute: On Alibaba Cloud, the MED attribute can have one of three values: 1000, 2000, or 3000. 1000 corresponds to an intra-region connection, 2000 corresponds to the default path, and 3000 corresponds to a cross-region connection.
MED priority: A lower value indicates a higher priority. When a TR or an ECR selects a path, it prioritizes the route entry with the lowest MED value.
MED propagation scope: The MED attribute is propagated only between ECRs and TRs and between TRs. It is not propagated to VBRs or VPCs.
MED propagation policy:
Outbound traffic direction: Outbound traffic is routed based on the route entries of the TR. When an ECR propagates a route entry to a TR, if the source VBR of the route entry and the receiving TR are in the same region, the MED value of the route entry is set to 1000. If they are in different regions, the MED value is set to 3000.
Inbound traffic direction: Inbound traffic is routed based on the route entries of the ECR. When a TR propagates a route entry to an ECR, if the route entry comes from a directly connected VPC in the same region, the TR sets the MED value of the route entry to 1000. If the route entry comes from another TR, the TR sets the MED value to 3000.
Route entry priority
Traffic path selection between TRs and ECRs is implemented based on the priority of route entries in their respective route tables.
Outbound route selection
Outbound traffic is routed based on the routes in a Transit Router (TR).
When a TR learns multiple routes to the same destination from ECRs and other TRs, it selects the best path based on TR route priorities. The TR forwards traffic using the route with the higher-priority attribute. If attributes are identical, the TR compares the next attribute in the sequence.
The route selection priorities are listed in the following table:
Note
The route attribute priorities are, from highest to lowest: P1 > P2 > P3 > P4.
|
Route attribute priority
|
Route attribute
|
Description
|
|
P1
|
AS_Path
|
The system compares the AS_Path of routes to the same destination.
A shorter AS_Path is preferred. If AS_Path lengths are the same, the system compares the local preference attribute.
|
|
P2
|
local preference
|
An intra-region next hop is preferred over a cross-region next hop.
-
If the next hop options include both an ECR and a cross-region TR, the ECR is preferred.
-
If all next hop options are TRs, the system compares the MED attribute.
|
|
P3
|
MED
|
When local preference is not the deciding factor, the MED value is compared. A lower MED value is preferred. You can set the MED of a target TR to the default value on the ECR to influence the path selection for outbound traffic. For more information, see Procedure.
If routes to the same destination have the same MED value, the system compares the byte order attribute.
|
|
P4
|
byte order
|
The system compares the region IDs of routes to the same destination. Region IDs are sorted alphabetically. The route from the region whose ID is first alphabetically receives higher priority.
For example, a route from the China (Beijing) (cn-beijing) region is preferred over a route from the China (Hangzhou) (cn-hangzhou) region.
|
Inbound route selection
Inbound traffic is routed based on the routes in an ECR.
When an ECR learns multiple routes to the same destination from TRs and other ECRs, it selects the best path by comparing the attributes of each route in descending order of priority. The ECR forwards traffic using the route with the higher-priority attribute. If attributes are identical, the ECR compares the next attribute in the sequence.
The route selection priorities are listed in the following table:
Note
The route attribute priorities are, from highest to lowest: P1 > P2 > P3 > P4.
|
Route attribute priority
|
Route attribute
|
Description
|
|
P1
|
MED
|
When an ECR learns routes to the same destination from TR instances, the MED is 1000 for routes from a TR in the same region, and 3000 for routes from a TR in a different region.
If all learned routes to the same destination are from cross-region TRs, their MED values are all 3000, so the system then compares the local preference attribute.
|
|
P2
|
local preference
|
If the region of a Virtual Border Router (VBR) has ECR and TR connections, traffic preferentially enters the cloud through the connections in the VBR's region.
If the VBR's region does not have ECR and TR connections, the system compares the default egress attribute.
|
|
P3
|
default egress
|
If the region of a VBR does not have ECR and TR connections, the system tags routes received from the TR that is set as the default egress. This ensures that the tagged route is preferred.
If no received TR route is tagged, the system compares the byte order attribute.
|
|
P4
|
byte order
|
The system compares the region IDs of routes to the same destination. Region IDs are sorted alphabetically. The route from the region whose ID is first alphabetically receives higher priority.
For example, a route from the China (Beijing) (cn-beijing) region is preferred over a route from the China (Hangzhou) (cn-hangzhou) region.
|
Principle examples
The following three scenarios, progressing from simple to complex, explain route selection between TRs and ECRs and demonstrate how to modify traffic paths. All scenarios assume that the AS_Path lengths are the same.
Scenario 1
In this scenario, a company's on-premises data center is connected to a Virtual Border Router (VBR) in the China (Hangzhou) region. The company has services deployed in Virtual Private Clouds (VPCs) in both the China (Hangzhou) and China (Shanghai) regions.
The company uses a VBR, an ECR, and TRs to connect the on-premises data center and the cloud VPCs. A cross-region connection exists between TR1 in Hangzhou and TR2 in Shanghai. The ECR connects to both TR1 and the VBR in Hangzhou.
1. Route propagation to VPCs
-
A: The data center advertises routes to the VBR.
-
B: The VBR advertises routes to the ECR.
-
C: The ECR advertises routes to the TR.
-
D: Routes are propagated between TRs. TR1 propagates the routes from the ECR to TR2.
2. Route propagation to data center
-
A: VPCs advertise routes to TRs. route learning must be enabled for the TRs.
-
B: Routes are propagated between TRs and to the ECR. TR1 propagates the routes from TR2 to the ECR.
-
C: The ECR advertises routes to the VBR.
-
D: The VBR advertises routes to the data center.
3. Traffic paths
Because there is only one path between the VBR and VPC1, and only one path between the VBR and VPC2, no route selection is necessary.
Scenario 2
In this scenario, a company has an on-premises data center in the China (Hangzhou) region, with services deployed in VPCs in the China (Hangzhou) and China (Shanghai) regions.
The company connects its on-premises data center and cloud VPCs by using a VBR, an ECR, and TRs. A cross-region connection links the TR in Hangzhou and the TR in Shanghai. The ECR connects to TR1 in Hangzhou, TR2 in Shanghai, and the VBR in Hangzhou, creating a fully interconnected network.
1. Route propagation to VPCs
-
A: The data center advertises routes to the VBR.
-
B: The VBR advertises routes to the ECR.
-
C: The ECR advertises routes to TR1 and TR2. The ECR sets the MED attribute on the routes before sending them to the two TRs.
When the source VBR and the destination TR are in the same region, the MED is set to 1000. When they are in different regions, the MED is set to 3000. Therefore:
-
Since the VBR and TR1 are in the same region, the ECR sets the MED of the route to 1000 before propagating it to TR1.
-
Since the VBR and TR2 are in different regions, the ECR sets the MED of the route to 3000 before propagating it to TR2.
-
D: Routes are propagated between TRs. After propagation, each TR determines its active routes as follows:
-
TR1 is directly connected to the ECR. The local preference rule prioritizes the route from the ECR. When outbound traffic from VPC1 accesses the VBR, TR1 selects the ECR as the next hop.
-
TR2 is also directly connected to the ECR. The local preference rule prioritizes the route from the ECR. When outbound traffic from VPC2 accesses the VBR, TR2 selects the ECR as the next hop.
2. Route propagation to data center
-
A: Routes are propagated between TRs.
-
B: TR1 and TR2 advertise routes to the ECR. The TRs set the MED attribute on the routes before sending them.
When the source VPC and the current TR are in the same region, the MED is set to 1000. When they are in different regions, the MED is set to 3000.
-
For routes originating from VPC1:
-
VPC1 and TR1 are in the same region. TR1 sets the MED to 1000 before propagating the route to the ECR.
-
VPC1 and TR2 are in different regions. TR2 sets the MED to 3000 before propagating the route to the ECR.
-
After the ECR receives the routes, it prefers the route from TR1 because it has a lower MED value. When inbound traffic accesses VPC1, the ECR selects TR1 as the next hop.
-
For routes originating from VPC2: The same logic applies. The ECR prefers the route from TR2. When inbound traffic accesses VPC2, the ECR selects TR2 as the next hop.
-
C: The ECR advertises routes to the VBR.
-
D: The VBR advertises routes to the data center.
3. Traffic paths
Based on the preceding route analysis:
-
Outbound traffic: On the TRs, since both are directly connected to the ECR, the local preference rule applies, and routes from the ECR are preferred. Therefore, each TR directly selects the ECR as its next hop, and traffic goes directly from the local TR to the ECR.
-
Inbound traffic: On the ECR, routes from the TR in the same region as the VPC are preferred. Therefore, traffic preferentially enters the cloud through the TR located in the same region as the destination VPC.
Scenario 3
In this scenario, a company has connected three on-premises data centers to VBRs in the China (Beijing), China (Shanghai), and China (Hangzhou) regions, respectively. Services are deployed in VPCs in these three regions.
The company has connected all three VBRs to an ECR, and has also connected TR1 and TR2 to the ECR. Cross-region connections link all three TRs.
1. Route propagation to VPCs
-
A: The data centers advertise routes to the VBRs.
-
B: The VBRs advertise routes to the ECR.
-
C: The ECR advertises routes to TR1 and TR2. The ECR sets the MED attribute on the routes before sending them.
When the source VBR and the destination TR are in the same region, the MED is set to 1000. When they are in different regions, the MED is set to 3000.
For a route originating from VBR1:
-
VBR1 and TR1 are in the same region. The ECR sets the MED to 1000 before propagating the route to TR1.
-
VBR1 and TR2 are in different regions. The ECR sets the MED to 3000 before propagating the route to TR2.
-
D: Routes are propagated between TRs. After propagation, each TR determines its active routes as follows:
-
TR1 and TR2 are directly connected to the ECR. This is the same as Scenario 2. Based on the local preference rule, the route from the ECR is preferred.
-
TR3 is not directly connected to the ECR. The logic is as follows:
-
When receiving routes originating from VBR1:
-
The route from TR1 has an MED of 1000.
-
The route from TR2 has an MED of 3000.
-
The route with an MED of 1000 is preferred. Therefore, the route from TR1 becomes active. When outbound traffic from VPC3 accesses VBR1, TR3 selects TR1 as the next hop.
-
When receiving routes originating from VBR2: The same logic applies. The route from TR2 becomes active. When outbound traffic from VPC3 accesses VBR2, TR3 selects TR2 as the next hop.
-
When receiving routes originating from VBR3:
-
For the route from TR1: The source VBR3 is in the China (Hangzhou) region, and TR1 is in the China (Beijing) region. This is a cross-region connection, so the MED is 3000.
-
For the route from TR2: The source VBR3 is in the China (Hangzhou) region, and TR2 is in the China (Shanghai) region. This is a cross-region connection, so the MED is 3000.
-
Because the MED values are the same, the byte order rule serves as a tie-breaker. The route from TR1 is prioritized because its region ID, cn-beijing, comes before cn-hangzhou alphabetically. When outbound traffic from VPC3 accesses VBR3, TR3 selects TR1 as the next hop.
Based on these routing rules, the outbound traffic paths from each VPC are as follows:
|
VPC1 outbound path
|
VPC2 outbound path
|
VPC3 outbound path
|
|
|
|
|
2. Route propagation to data centers
-
A: VPCs advertise routes to TRs.
-
B: Routes are propagated between TRs and to the ECR. The TRs set the MED attribute on the routes before sending them to the ECR.
When the source VPC and the current TR are in the same region, the MED is set to 1000. When they are in different regions, the MED is set to 3000.
-
For routes originating from VPC1:
-
VPC1 and TR1 are in the same region. TR1 sets the MED to 1000 before propagating the route to the ECR.
-
VPC1 and TR2 are in different regions. TR2 sets the MED to 3000 before propagating the route to the ECR.
-
After the ECR receives the routes, it prefers the route from TR1 because it has a lower MED value. When inbound traffic accesses VPC1, the ECR selects TR1 as the next hop.
-
For routes originating from VPC2: The same logic applies. The ECR prefers the route from TR2. When inbound traffic accesses VPC2, the ECR selects TR2 as the next hop.
-
For routes originating from VPC3:
-
VPC3 and TR1 are in different regions. TR1 sets the MED to 3000 before propagating the route to the ECR.
-
VPC3 and TR2 are in different regions. TR2 sets the MED to 3000 before propagating the route to the ECR.
-
Because the MED values of the routes received by the ECR are the same, the local preference rule applies:
-
The TR in the same region as VBR1 is TR1, which is directly connected to the ECR. Therefore, for inbound traffic from VBR1 to VPC3, the ECR selects TR1 as the next hop.
-
The TR in the same region as VBR2 is TR2, which is directly connected to the ECR. Therefore, for inbound traffic from VBR2 to VPC3, the ECR selects TR2 as the next hop.
-
The TR in the same region as VBR3 is TR3, which is not directly connected to the ECR. The byte order rule is used as a tie-breaker:
The cn-beijing region ID comes before cn-hangzhou alphabetically, which prioritizes the route from TR1. For inbound traffic from VBR3 to VPC3, the ECR selects TR1 as the next hop.
-
C: The ECR advertises routes to the VBRs.
-
D: The VBRs advertise routes to the data centers.
Based on these routing rules, the inbound traffic paths from the data centers to the three VPCs are as follows:
|
VPC1 inbound path
|
VPC2 inbound path
|
VPC3 inbound path
|
|
|
|
|
3. Change the traffic path
You can set TR2 as the default path on the ECR to change the path for traffic between VPC3 and data center 3:
-
Change the outbound traffic path:
After TR2 is set as the default path, the ECR sets the MED attribute to 2000 for all routes sent to TR2. When TR2 propagates these routes to TR3, TR3 prefers the routes from TR2. This changes the outbound traffic egress for VPC3 from TR1 to TR2.
-
Change the inbound traffic path:
After TR2 is set as the default path, the default egress rule applies. Inbound traffic preferentially selects the TR set as default (TR2) as the next hop. Now, when inbound traffic from VBR3 accesses VPC3, the ECR selects TR2 as the next hop.
|
Path before MED change
|
Path after MED change
|
|
|
|
The following table summarizes the traffic paths between the three VPCs and their corresponding data centers.
|
MED status
|
VPC1-DC1 path
|
VPC2-DC2 path
|
VPC3-DC3 path
|
|
Before the change
|
|
|
|
|
After the change
|
Path unchanged
|
Path unchanged
|
|