When you connect transit routers (TRs) in multiple regions of a Cloud Enterprise Network (CEN) to an Express Connect Router (ECR), multiple traffic paths are created for communication between on-premises and cloud resources. This topic describes the routing rules in this scenario.
Routing rules
First, the system determines the outbound traffic path. By default, when outbound traffic from a Virtual Private Cloud (VPC) reaches a TR:
If the current TR is directly connected to the ECR, the traffic is routed outbound through the current TR to the ECR.
If the current TR is not directly connected to the ECR, the traffic is routed outbound through another TR that is directly connected to the ECR. If multiple such TRs exist:
The system selects the TR in the same region as the destination Virtual Border Router (VBR).
If none of the other TRs are in the same region as the destination VBR, the system selects the TR in the region with the lexicographically smallest region ID.
After the outbound traffic path is determined, the system ensures that the inbound traffic path is symmetrical to the outbound traffic path.
Scenario examples
This section provides two examples to help you understand the routing rules.
Example 1: Multiple TRs are connected to an ECR
In the scenario shown in the preceding figure, an enterprise has a data center in the China (Hangzhou) region and has deployed services in VPCs in the China (Hangzhou) and China (Shanghai) regions.
The enterprise uses a VBR, an ECR, and TRs to connect the on-premises data center to the cloud VPCs. An inter-region connection is established between the TR in Hangzhou and the TR in Shanghai. The ECR is attached to TR1 in Hangzhou, TR2 in Shanghai, and the VBR in Hangzhou.
1. Outbound traffic path
When VPC1 accesses the on-premises data center, traffic from VPC1 reaches TR1. Because TR1 is directly connected to the ECR, the traffic is routed outbound directly from TR1 to the ECR.
When VPC2 accesses the on-premises data center, traffic from VPC2 reaches TR2. Because TR2 is also directly connected to the ECR, the traffic is routed outbound directly from TR2 to the ECR.
2. Inbound traffic path
After the outbound traffic paths are determined, the system directs inbound traffic to follow symmetrical paths:
When the data center accesses VPC1, traffic reaches the ECR and then enters VPC1 through TR1.
When the data center accesses VPC2, traffic reaches the ECR and then enters VPC2 through TR2.
Example 2: Multiple TRs are interconnected, but some are not connected to the ECR
In the scenario shown in the preceding figure, an enterprise has connected three data centers to VBRs in the China (Beijing), China (Shanghai), and China (Hangzhou) regions. The enterprise has also deployed services in VPCs in these three regions.
The enterprise has connected the three VBRs to an ECR and connected TR1 and TR2 to the ECR. Inter-region connections are established among all three TRs.
1. Outbound traffic path
Outbound traffic path from VPC1 | Outbound traffic path from VPC2 | Outbound traffic path from VPC3 |
 |  |  |
When VPC1 sends outbound traffic to access the three data centers, the traffic is routed outbound directly from TR1 to the ECR because TR1 is directly connected to the ECR.
The path for outbound traffic from VPC2 is similar to that from VPC1. The traffic is routed outbound directly from TR2 to the ECR.
When VPC3 sends outbound traffic to access the three data centers, the traffic is routed to either TR1 or TR2 because TR3 is not directly connected to the ECR:
When VPC3 accesses IDC1, the traffic reaches TR3 and is then routed outbound through TR1, which is in the same region as the destination VBR1.
When VPC3 accesses IDC2, the traffic reaches TR3 and is then routed outbound through TR2, which is in the same region as the destination VBR2.
When VPC3 accesses IDC3, the traffic reaches TR3. Because neither TR1 nor TR2 is in the same region as the destination VBR3, the traffic is routed outbound through TR1. This is because the region ID of TR1 (cn-beijing) is lexicographically smaller than the region ID of TR2 (cn-shanghai).
2. Inbound traffic path
Inbound traffic path to VPC1 | Inbound traffic path to VPC2 | Inbound traffic path to VPC3 |
 |  |  |
After the outbound traffic paths are determined, the system directs inbound traffic to follow symmetrical paths. The preceding figure shows these details, so they are not repeated here.
Modify the default routing behavior
In scenarios where a TR is selected based on the lexicographical order of region IDs, you can manually specify a TR to change the default traffic path. By changing the traffic path, you can achieve the following:
Reduce latency: Change geographically circuitous traffic paths to reduce network latency.
Reduce costs: Prioritize low-cost inter-region links for service traffic to reduce expenses because the bandwidth costs for different inter-region connections in the cloud network may vary.
Control traffic distribution: If a TR in a specific region experiences high traffic, you can direct some service traffic to a less congested TR to improve overall network quality and throughput.
Default traffic path between VPC3 and IDC3 | Traffic path between VPC3 and IDC3 after modification |
 |  |
As shown in the preceding figure, when VPC3 accesses IDC3, the system compares the region IDs of TR1 and TR2 according to the routing rules. By default, traffic is transmitted through TR1.
You can set TR2 as the default path to route traffic between VPC3 and IDC3 through TR2, thereby changing the default traffic path.
Procedure
Go to the Basic Information page of the destination ECR. Find the destination TR and click Set As Default in the Actions column.
How it works
The routing rules between a TR and an ECR are based on the priority of routes in their respective route tables. The following section explains the implementation principles.
MED description
Multi-Exit Discriminator (MED) is a path attribute of Border Gateway Protocol (BGP). When multiple paths for the same route exist between contiguous autonomous systems, you can set the MED attribute to change the traffic path.
Location of the MED attribute: The MED attribute is stored in a route entry. You can view it in the MED column of the ECR route entry list.
Value range of the MED attribute: On Alibaba Cloud, the MED attribute can have three values: 1000, 2000, and 3000. 1000 corresponds to the same region, 2000 corresponds to default, and 3000 corresponds to an inter-region connection.
MED priority: A smaller value indicates a higher priority. When a TR or an ECR selects a route, it prioritizes the route with the lower MED value.
MED propagation scope: The MED attribute is propagated only between ECRs and TRs and between TRs. It is not propagated to VBRs or VPCs.
MED propagation policy:
Outbound traffic direction: Outbound traffic is routed based on the routes in the TR. When an ECR propagates a route to a TR, if the source VBR and the receiving TR are in the same region, the MED value of the route is set to 1000. If they are in different regions, the MED value is set to 3000.
Inbound traffic direction: Inbound traffic is routed based on the routes in the ECR. When a TR propagates a route to an ECR, if the route originates from a directly connected VPC in the same region, the TR sets the MED value of the route to 1000. If the route originates from another TR, the TR sets the MED value of the route to 3000.
Route priority
Traffic routing between a TR and an ECR is determined by the priority of routes in their respective route tables.
Route selection rules for outbound traffic
Outbound traffic is routed based on the routes of an Enterprise Edition transit router (TR).
When a TR learns routes to the same destination CIDR block from ECR instances and other TR instances, the system compares the value of each property in descending order of priority based on the TR route priority. The route with the higher-priority property value is selected to forward traffic. If multiple routes have the same value for a property, the system compares the value of the next property.
The route selection priorities are listed in the following table:
The route attribute priorities are, from highest to lowest: P1 > P2 > P3 > P4.
Route attribute priority | Route attribute | Attribute value description |
P1 | AS_Path | The system compares the AS_Path of routes to the same destination CIDR block. A shorter AS_Path is preferred. If the AS_Path lengths are the same, the local preference attribute is compared. |
P2 | Local preference | A next hop to an intra-region connection has a higher priority than a next hop to an inter-region connection.
|
P3 | MED | If the VPC is in a region that does not have a connection to an ECR and a TR, the MED values are compared. The route with the smaller MED value is preferred. You can set the MED value of the destination TR to Default in the ECR to change the path selection for outbound traffic. For more information, see Procedure. If routes to the same destination CIDR block have the same MED value, the lexicographical order attribute is compared. |
P4 | Byte order | The system compares the region IDs of the routes to the same destination CIDR block. The region IDs are sorted alphabetically. The route from the region ID that comes first alphabetically has a higher priority. For example, a route from the China (Beijing) (cn-beijing) region has a higher priority than a route from the China (Hangzhou) (cn-hangzhou) region. |
Route selection rules for inbound traffic
Inbound traffic is routed based on the routes of an ECR.
When an ECR learns routes to the same destination CIDR block from TR instances and other ECR instances, the system compares the attribute values of each route. The comparison follows the route priority from highest to lowest. The route with the higher-priority attribute is used to forward traffic. If multiple routes have the same value for an attribute, the system compares the next attribute.
The route selection priorities are listed in the following table:
The route attribute priorities are, from highest to lowest: P1 > P2 > P3 > P4.
Route attribute priority | Route attribute | Attribute value description |
P1 | MED | An ECR learns routes to the same destination CIDR block from TR instances. The MED value is 1000 for routes from a TR in the same region. The MED value is 3000 for routes from a TR in a different region. If all learned routes to the same destination CIDR block are from inter-region TRs, their MED values are all 3000. The local preference attribute is then compared. |
P2 | Local preference | If the region of the virtual border router (VBR) has connections to an ECR and a TR, inbound traffic preferentially enters the cloud through the ECR and TR connections in the same region as the VBR. If the VBR's region does not have connections to an ECR and a TR, the default egress tag attribute is compared. |
P3 | Default egress | If the VBR's region does not have connections to an ECR and a TR, the system tags the TR route received from the connection that is set as the default egress. This ensures that the tagged route is preferred when it enters the ECR. If the TR route received by the ECR egress connection is not tagged, the lexicographical order attribute is compared. |
P4 | Byte order | The system compares the region IDs of the routes to the same destination CIDR block. The region IDs are sorted alphabetically. The route from the region ID that comes first alphabetically has a higher priority. For example, a route from the China (Beijing) (cn-beijing) region has a higher priority than a route from the China (Hangzhou) (cn-hangzhou) region. |
Principle examples
The following three scenarios are described in increasing order of complexity. These scenarios explain the route selection principles between TRs and ECRs and how to change traffic paths. All scenarios assume that the AS_Path lengths are the same.
Scenario 1: Multiple TRs, but only one is connected to the ECR
As shown in the figure, a company's data center (IDC) is connected to a virtual border router (VBR) in the China (Hangzhou) region. The company has services deployed in VPCs in the China (Hangzhou) and China (Shanghai) regions.
The company uses a VBR, an ECR, and TRs to connect the on-premises IDC to the cloud VPCs. An inter-region connection is established between TR1 in Hangzhou and TR2 in Shanghai. The ECR is attached to TR1 and the VBR, both of which are in the Hangzhou region.
1. Route propagation from the IDC to the VPCs
A: The IDC propagates routes to the VBR.
B: The VBR propagates routes to the ECR.
C: The ECR propagates routes to the TR.
D: Routes are propagated between TRs. TR1 propagates the routes from the ECR to TR2.
2. Route propagation from the VPCs to the IDC
A: The VPCs propagate routes to the TRs. (Route learning is enabled on the TRs.) For more information, see Enable route learning.
B: Routes are propagated between TRs and to the ECR. TR1 propagates the routes from TR2 to the ECR.
C: The ECR propagates routes to the VBR.
D: The VBR propagates routes to the IDC.
3. Traffic paths
Because there is only one path between the VBR and VPC1, and one path between the VBR and VPC2, no route selection is needed. The traffic paths are shown in the figure.
Scenario 2: Multiple TRs are connected to the ECR
As shown in the figure, a company has an IDC in the China (Hangzhou) region and services deployed in VPCs in the China (Hangzhou) and China (Shanghai) regions.
The company uses a VBR, an ECR, and TRs to connect the on-premises IDC to the cloud VPCs. An inter-region connection is established between the TR in Hangzhou and the TR in Shanghai. The ECR is attached to TR1 in Hangzhou, TR2 in Shanghai, and the VBR in Hangzhou. The TRs and the ECR are fully interconnected.
1. Route propagation from the IDC to the VPCs
A: The IDC propagates routes to the VBR.
B: The VBR propagates routes to the ECR.
C: The ECR propagates routes to TR1 and TR2. The ECR first sets the MED attribute in the routes and then sends the routes to the two TRs.
When the source VBR and the destination TR of a route are in the same region, the MED value is set to 1000. When they are in different regions, the MED value is set to 3000. Therefore:
The VBR and TR1 are in the same region. The ECR sets the MED value of the route to 1000 and then propagates it to TR1.
The VBR and TR2 are in different regions. The ECR sets the MED value of the route to 3000 and then propagates it to TR2.
D: Routes are propagated between TRs. After propagation, the active routes on each TR are as follows:
TR1 is directly connected to the ECR. Based on the intra-region priority rule, the route from the ECR takes precedence. When outbound traffic from VPC1 accesses the VBR, TR1 selects the ECR as the next hop.
TR2 is also directly connected to the ECR. Based on the intra-region priority rule, the route from the ECR takes precedence. When outbound traffic from VPC2 accesses the VBR, TR2 selects the ECR as the next hop.
2. Route propagation from the VPCs to the IDC
A: Routes are propagated between TRs.
B: TR1 and TR2 propagate routes to the ECR. The TRs first set the MED attribute in the routes and then send the routes to the ECR.
When the source VPC and the current TR of a route are in the same region, the MED value is set to 1000. When they are in different regions, the MED value is set to 3000.
For routes originating from VPC1:
VPC1 and TR1 are in the same region. TR1 sets the MED value of the route to 1000 and propagates it to the ECR.
VPC1 and TR2 are in different regions. TR2 sets the MED value of the route to 3000 and propagates it to the ECR.
After the ECR receives the routes, the route with the lower MED value has a higher priority. The route from TR1 takes precedence. When inbound traffic accesses VPC1, the ECR selects TR1 as the next hop.
For routes originating from VPC2: The logic is the same as for routes from VPC1. The ECR gives precedence to the route from TR2. When inbound traffic accesses VPC2, the ECR selects TR2 as the next hop.
C: The ECR propagates routes to the VBR.
D: The VBR propagates routes to the IDC.
3. Traffic paths
Based on the preceding route analysis:
Outbound traffic: On the TRs, the routes from the ECR take precedence based on the intra-region priority rule because both TRs are directly connected to the ECR. Therefore, the TRs select the ECR as the next hop. Traffic is sent outbound directly from the local TR to the ECR.
Inbound traffic: On the ECR, the route from the TR in the same region as the source VPC has a higher priority. Therefore, traffic preferentially enters the cloud through the TR in the VPC's region.
Scenario 3: Multiple TRs are interconnected, and one TR is not connected to the ECR
As shown in the figure, a company has connected three IDCs to VBRs in the China (Beijing), China (Shanghai), and China (Hangzhou) regions. The company also has services deployed in VPCs in the China (Beijing), China (Shanghai), and China (Hangzhou) regions.
The company has connected the three VBRs to an ECR and connected TR1 and TR2 to the ECR. Inter-region connections are established among all three TRs.
1. Route propagation from the IDCs to the VPCs
A: The IDCs propagate routes to the VBRs.
B: The VBRs propagate routes to the ECR.
C: The ECR propagates routes to TR1 and TR2. The ECR first sets the MED attribute in the routes and then sends the routes to the TRs.
When the source VBR and the destination TR of a route are in the same region, the MED value is set to 1000. When they are in different regions, the MED value is set to 3000.
Take the route originating from VBR1 as an example:
VBR1 and TR1 are in the same region. The ECR sets the MED value of the route to 1000 and then propagates it to TR1.
VBR1 and TR2 are in different regions. The ECR sets the MED value of the route to 3000 and then propagates it to TR2.
D: Routes are propagated between TRs. After propagation, the active routes on each TR are as follows:
TR1 and TR2 are both directly connected to the ECR. The situation is the same as in Scenario 2. Based on the intra-region priority rule, the routes from the ECR take precedence.
TR3 is not directly connected to the ECR. The situation is as follows:
When receiving routes originating from VBR1:
Route from TR1: The MED value is 1000.
Route from TR2: The MED value is 3000.
The route with an MED of 1000 has a higher priority. The route from TR1 takes precedence. When outbound traffic from VPC3 accesses VBR1, TR3 selects TR1 as the next hop.
When receiving routes originating from VBR2, the logic is similar to that for VBR1. The route from TR2 takes precedence. When outbound traffic from VPC3 accesses VBR2, TR3 selects TR2 as the next hop.
When receiving routes originating from VBR3:
Route from TR1: The source VBR3 is in Hangzhou and TR1 is in Beijing. This is an inter-region connection, so the MED value is 3000.
Route from TR2: The source VBR3 is in Hangzhou and TR2 is in Shanghai. This is an inter-region connection, so the MED value is 3000.
The MED values are the same. Priority is determined by lexicographical order. `beijing` comes before `hangzhou` alphabetically, so the route from TR1 takes precedence. When outbound traffic from VPC3 accesses VBR3, TR3 selects TR1 as the next hop.
Based on the preceding routing rules, the outbound traffic paths from each VPC are as follows:
Outbound traffic path from VPC1 | Outbound traffic path from VPC2 | Outbound traffic path from VPC3 |
 |  |  |
2. Route propagation from the VPCs to the IDCs
A: The VPCs propagate routes to the TRs.
B: Routes are propagated between TRs and to the ECR. The TRs first set the MED attribute in the routes and then send the routes to the ECR.
When the source VPC and the current TR of a route are in the same region, the MED value is set to 1000. When they are in different regions, the MED value is set to 3000.
For routes originating from VPC1:
VPC1 and TR1 are in the same region. TR1 sets the MED value of the route to 1000 and propagates it to the ECR.
VPC1 and TR2 are in different regions. TR2 sets the MED value of the route to 3000 and propagates it to the ECR.
After the ECR receives the routes, the route with the lower MED value has a higher priority. The route from TR1 takes precedence. When inbound traffic accesses VPC1, the ECR selects TR1 as the next hop.
For routes originating from VPC2: The logic is the same as for routes from VPC1. The ECR gives precedence to the route from TR2. When inbound traffic accesses VPC2, the ECR selects TR2 as the next hop.
For routes originating from VPC3:
VPC3 and TR1 are in different regions. TR1 sets the MED value of the route to 3000 and propagates it to the ECR.
VPC3 and TR2 are in different regions. TR2 sets the MED value of the route to 3000 and propagates it to the ECR.
The ECR receives routes with the same MED value. The intra-region priority logic is applied:
The TR in the same region as VBR1 is TR1, which is directly connected to the ECR. Therefore, when inbound traffic from VBR1 accesses VPC3, the ECR selects TR1 as the next hop.
The TR in the same region as VBR2 is TR2, which is directly connected to the ECR. Therefore, when inbound traffic from VBR2 accesses VPC3, the ECR selects TR2 as the next hop.
The TR in the same region as VBR3 is TR3, which is not directly connected to the ECR. Lexicographical comparison is used:
`beijing` comes before `hangzhou` alphabetically, so the route from TR1 takes precedence. When inbound traffic from VBR3 accesses VPC3, the ECR selects TR1 as the next hop.
C: The ECR propagates routes to the VBRs.
D: The VBRs propagate routes to the IDCs.
Based on the preceding routing rules, the inbound traffic paths from the IDCs to the three VPCs are as follows:
Inbound traffic path to VPC1 | Inbound traffic path to VPC2 | Inbound traffic path to VPC3 |
 |  |  |
3. Changing traffic paths
You can set TR2 as the default path on the ECR to change the traffic path for communication between VPC3 and IDC3:
Change the outbound traffic path:
After you set TR2 as the default path on the ECR, the MED attribute of all routes sent from the ECR to TR2 is set to 2000 based on the MED propagation policy. After TR2 sends these routes to TR3, TR3 gives precedence to the routes from TR2. This changes the outbound traffic egress for VPC3 from TR1 to TR2.
Change the inbound traffic path:
After you set TR2 as the default path on the ECR, inbound traffic preferentially selects the default TR2 as the next hop based on the default egress rule. As a result, when inbound traffic from VBR3 accesses VPC3, the ECR selects TR2 as the next hop.
Traffic path between VPC3 and IDC3 before changing the MED | Traffic path between VPC3 and IDC3 after changing the MED |
 |  |
The following table summarizes the traffic paths for communication between the three VPCs and the IDCs:
MED modification status | Traffic path between VPC1 and IDC | Traffic path between VPC2 and IDC | Traffic path between VPC3 and IDC |
Before changing the MED |  |  |  |
After changing the MED | The traffic path remains unchanged | The traffic path remains unchanged |  |