This topic describes how to use Border Gateway Protocol (BGP) dynamic routing with a virtual border router (VBR) and a private IPsec-VPN connection to encrypt private traffic over an Express Connect circuit.
Background information
Before you start, we recommend that you learn about how private traffic is encrypted and transmitted over Express Connect circuits. For more information, see Encrypt private connections over Express Connect circuits.
Example scenario
The following figure shows a use scenario. An enterprise owns a data center in Hangzhou and has a VPC deployed in the China (Hangzhou) region. Applications are deployed on Elastic Compute Service (ECS) instances in the VPC. To expand businesses, the enterprise wants to connect the data center to the cloud. To meet security compliance requirements, the enterprise needs to use Express Connect circuits and transit routers to exchange data between the data center and VPC through private connections. In addition, to prevent data breaches and data tampering, the enterprise wants to encrypt data before transmitting the data to Alibaba Cloud over the Express Connect circuit.
After private connectivity is established between the data center and the VPC, the enterprise can create a private IPsec-VPN connection between the on-premises gateway device and the transit router. The private IPsec-VPN connection encrypts the traffic that passes through the Express Connect circuit to meet the enterprise's stringent security requirements.
Network planning
When you plan CIDR blocks for your data center and related network instances, make sure that the CIDR blocks of the networks that need to communicate with each other do not overlap.
Routing mechanism
To encrypt private traffic over an Express Connect circuit, you must ensure that traffic between the data center and the VPC is preferentially transmitted through the private IPsec-VPN connection, not the Express Connect circuit. This guide achieves this goal by controlling routes:
Traffic from the VPC to the data center:
The transit router can learn routes to the data center from both the VBR instance and the private IPsec-VPN connection. Based on the route priorities of the transit router, the route learned from the VBR instance is preferred by default. This behavior prevents encryption because traffic is preferentially transmitted over the unencrypted Express Connect circuit.
This guide avoids this issue by advertising the data center CIDR block with different subnet masks. When the on-premises gateway device advertises the data center CIDR block to the VBR instance, it must advertise an aggregate CIDR block (with a shorter subnet mask). When it advertises the data center CIDR block to the private IPsec-VPN connection, it must advertise a more specific CIDR block (with a longer subnet mask).
For example, if the CIDR block of the data center is 192.168.0.0/16 and the client CIDR block in the data center that communicates with the VPC is 192.168.20.0/24, the on-premises gateway device advertises the data center CIDR block 192.168.0.0/16 to the VBR instance. It advertises the specific client CIDR block 192.168.20.0/24 to the private IPsec-VPN connection. This ensures that the transit router prefers the route learned through the private IPsec-VPN connection. Traffic from the VPC to the client is preferentially encrypted and transmitted over the private IPsec-VPN connection.
Traffic from the data center to the VPC:
The data center can learn routes to the VPC from both the VBR instance and the private IPsec-VPN connection. In this guide, a routing policy is configured on the transit router to adjust route priorities. This ensures that traffic from the data center to the VPC is preferentially transmitted over the private IPsec-VPN connection.
This ensures that the data center and VPC can still exchange data through the Express Connect circuit and transit router when the private IPsec-VPN connection is closed. However, the data is not encrypted in this case.
Basic network planning
Configuration objective | Network planning | IP address |
VPC |
|
|
VBR | 10.0.0.0/30 |
|
Data center | Client CIDR block: 192.168.20.0/24 | Client address: 192.168.20.6 |
On-premises gateway device CIDR blocks:
|
|
BGP network planning
The BGP tunnel CIDR block must be a /30 subnet within 169.254.0.0/16. It cannot be 169.254.0.0/30, 169.254.1.0/30, 169.254.2.0/30, 169.254.3.0/30, 169.254.4.0/30, 169.254.5.0/30, 169.254.6.0/30, or 169.254.169.252/30. The two tunnels of an IPsec-VPN connection cannot use the same tunnel CIDR block.
Resource | Tunnel | BGP tunnel CIDR block | BGP IP address | BGP AS number (local autonomous system number) |
IPsec-VPN connection instance | Tunnel 1 | 169.254.10.0/30 | 169.254.10.1 | 65534 |
Tunnel 2 | 169.254.20.0/30 | 169.254.20.1 | ||
On-premises gateway device | Tunnel 1 | 169.254.10.0/30 | 169.254.10.2 | 65530 |
Tunnel 2 | 169.254.20.0/30 | 169.254.20.2 |
Prerequisites
A VPC is created in the China (Hangzhou) region and applications are deployed on the ECS instances in the VPC. For more information, see Create a VPC with an IPv4 CIDR block.
The customer gateway device supports the IKEv1 and IKEv2 protocols for establishing private IPsec-VPN connections. To check whether the gateway device supports the IKEv1 and IKEv2 protocols, contact the gateway vendor.
Procedure
Step 1: Use an Express Connect circuit and a transit router to establish private connectivity between the data center and the VPC
Substep 1: Deploy an Express Connect circuit
You can deploy an Express Connect circuit to connect the data center to Alibaba Cloud.
Create a connection over an Express Connect circuit.
Apply for an Express Connect circuit in the China (Hangzhou) region. For more information, see Classic mode or Overview of hosted connections. This example creates A Dedicated Connection Over An Express Connect Circuit.
Create a VBR instance.
Log on to the Express Connect console.
In the navigation pane on the left, click Virtual Border Routers (VBRs).
In the top navigation bar, select the China (Hangzhou) region.
The VBR instance must be in the same region as the Express Connect circuit.
On the Virtual Border Routers (VBRs) page, click Create VBR.
In the Create VBR panel, set the following parameters and click OK.
The following table describes only some of the parameters. Use the default values for the other parameters. For more information, see Create and manage a VBR.
Configuration item
Description
Name
Enter VBR.
Express Connect Circuit
In this example, select Dedicated Physical Connection and then select an existing Express Connect circuit.
VLAN ID
Enter 0.
Alibaba Cloud-side Peer IPv4 Address
Enter 10.0.0.1.
Customer-Side IPv4 Peer IP Address
Enter 10.0.0.2.
IPv4 Subnet Mask
Enter 255.255.255.252.
Configure a BGP group for the VBR instance.
On the Virtual Border Routers (VBRs) page, click the VBR ID.
On the details page, click the BGP Groups tab.
On the BGP Groups tab, click Create BGP Group, set the following parameters, and click OK.
The following table describes only the relevant parameters. For more information, see Configure BGP.
In the Name field, enter VBR-BGP.
Peer ASN: The ASN of the on-premises gateway device. For example, 65530.
Local ASN: Enter 65534 as the BGP ASN of the VBR.
Configure a BGP peer for the VBR instance.
On the VBR details page, click the BGP Peers tab.
On the BGP Peers tab, click Create BGP Peer.
In the Create BGP Peer panel, configure the BGP peer and click OK.
BGP Group: Select VBR-BGP.
BGP Peer IP Address: Enter the IP address of the interface on the on-premises gateway device that connects to the Express Connect circuit. In this topic, enter 10.0.0.1.
Configure BGP routing on the on-premises gateway device.
NoteIn this example, the software Adaptive Security Appliance (ASA) 9.19.1 is used to describe how to configure a Cisco firewall. The commands may vary with software versions. Consult the documentation or your vendor based on your actual environment during operations. For more information, see Configure local gateways.
The following content contains third-party product information, which is only for reference. Alibaba Cloud does not make guarantees or other forms of commitments for the performance and reliability of third-party products, or the potential impacts of operations performed by using these products.
ciscoasa> enable Password: ******** # Enter the password for entering the enable mode. ciscoasa# configure terminal # Enter the configuration mode. ciscoasa(config)# # Verify that the interfaces are configured and enabled on the Cisco firewall. In this example, the following interface configurations are used: ciscoasa(config)# show running-config interface ! interface GigabitEthernet0/0 # The interface that connects to the VBR. nameif VBR # The name of the GigabitEthernet 0/0 interface. security-level 0 ip address 10.0.0.1 255.255.255.252 # The IP address of the GigabitEthernet0/0 interface. ! interface GigabitEthernet0/2 # The interface that connects to the data center. nameif private # The name of the GigabitEthernet 0/2 interface. security-level 100 # Make sure that the security level of the interface connecting to the data center is lower than that of the interface connecting to Alibaba Cloud. ip address 192.168.50.215 255.255.255.0 # The IP address of the GigabitEthernet0/2 interface. ! interface GigabitEthernet0/3 # The interface that connects to the private IPsec-VPN tunnel 1. nameif VPN-IP1 # The name of the GigabitEthernet0/3 interface. security-level 0 ip address 192.168.10.136 255.255.255.0 # The private IP address of the GigabitEthernet0/3 interface. ! interface GigabitEthernet0/4 # The interface that connects to the private IPsec-VPN tunnel 2. nameif VPN-IP2 # The name of the GigabitEthernet0/4 interface. security-level 0 ip address 192.168.40.159 255.255.255.0 # The private IP address of the GigabitEthernet0/4 interface. ! # Configure prefix-list and route-map. prefix-list VBR permit 192.168.0.0/16 route-map VBR permit 10 match ip address prefix-list VBR # Configure BGP routing. router bgp 65530 # Enable BGP and configure the ASN of the data center. 65530 is used in this example. bgp router-id 10.0.0.1 # Enter the ID of the BGP router. In this example, 10.0.0.1 is used. address-family ipv4 unicast neighbor 10.0.0.2 remote-as 65534 # Establish a peering connection to the VBR. neighbor 10.0.0.2 activate # Activate the BGP peer. neighbor 10.0.0.2 route-map VBR out # Advertise only routes pointing to large CIDR blocks to the VBR. network 192.168.0.0 mask 255.255.0.0 # Advertise the CIDR block of the data center. We recommend that you use a short subnet mask. exit-address-family ! # Add a route that points to the client in the data center. route private 192.168.0.0 255.255.0.0 192.168.50.216ImportantWhen you advertise the data center CIDR block to the VBR instance, we recommend that you advertise an aggregate CIDR block. This ensures that the route learned by the transit router through the private IPsec-VPN connection is more specific, and therefore has a higher priority, than the route advertised by the VBR.
Substep 2: Configure a transit router
After the data center is connected to Alibaba Cloud through the Express Connect circuit, you can configure a transit router to enable private communication between the data center and the VPC.
In the Create CEN Instance dialog box, click Create CEN Only, enter a name, and use the default settings for other parameters.
Create an Enterprise Edition transit router.
Create a transit router in the China (Hangzhou) region to connect the VBR and VPC. Use the default settings for other parameters.
Create a VPC connection.
On the CEN details page, click the tab. Find the transit router in the China (Hangzhou) region and click Create Connection in the Actions column.
On the Connection with Peer Network Instance page, configure the following parameters and click OK to connect the VPC to the transit router.
The following table describes only some of the parameters. Other parameters use the default values. For more information, see Use an Enterprise Edition transit router to create a VPC connection.
Parameter
Description
Instance Type
Select Virtual Private Cloud (VPC).
Region
Select China (Hangzhou).
Attachment Name
Enter VPC-Attachment.
Network Instance
Select your VPC.
VSwitch
Select the vSwitches that are deployed in the zones of the transit router.
In this example, vSwitch 2 and vSwitch 3 are selected. If the region has multiple zones, choose at least two zones and select one vSwitch in each zone. We recommend that you select idle vSwitches.
Advanced Settings
In this example, the default settings are used. All advanced features are enabled.
Click Create More Connections to return to the Connection with Peer Network Instance page.
Create a VBR connection.
On the Connection with Peer Network Instance page, configure the following parameters and click OK to connect the VBR to the transit router. The following table describes only some of the parameters. Other parameters use the default values. For more information, see Connect a VBR to an Enterprise Edition transit router.
Parameter
Description
Instance Type
Select Virtual Border Router (VBR).
Region
Select China (Hangzhou).
Attachment Name
VBR-Attachment is entered in this example.
Network Instance
VBR is selected in this example.
Advanced Settings
In this example, the default settings are used. All advanced features are enabled.
Substep 3: Test the private connectivity
After you complete the preceding configurations, private connectivity is established between the data center and the VPC. You can test the private connectivity as follows.
Make sure that you are familiar with the security group rules for the ECS instances in the VPC and the access control rules for the client in the data center. Make sure that the rules allow the ECS instances in the VPC to communicate with the client in the data center. For more information, see View security group rules and Add a security group rule.
The access control rules for the data center must allow ICMP messages and access from the VPC. The security group rules for the ECS instances must allow ICMP messages and access from the CIDR block of the data center.
Connect to ECS 1 in the VPC. For more information, see Connection method overview.
Run the ping command to ping a client in the data center.
ping <the IP address of a client in the data center>
As displayed in the preceding figure, if ECS 1 can receive the response, the data center and VPC are connected.
Step 2: Encrypt private traffic over the Express Connect circuit
After private connectivity is established between the data center and the VPC, you can create a private IPsec-VPN connection between the on-premises gateway device and the transit router. Then, you can configure routes to direct traffic between the data center and the VPC through the private IPsec-VPN connection to encrypt the private traffic.
Substep 1: Establish a private IPsec-VPN connection
Add the CIDR block 10.10.10.0/24 for the transit router. For more information, see Transit router CIDR blocks.
A gateway IP address will be allocated from the transit router CIDR block to create a private IPsec-VPN connection. The transit router CIDR block must not overlap with the CIDR blocks of the data center and VPC used for communication.
Create two customer gateways to register the two VPN IP addresses and the BGP AS number of the on-premises gateway device with Alibaba Cloud.
- Log on to the VPN gateway console.
In the left navigation pane, select .
On the Customer Gateway page, click Create Customer Gateway.
In the Create Customer Gateway panel, set the following parameters and click OK.
This guide describes only some of the parameters. Use the default values for the other parameters. For more information, see Create and manage a customer gateway.
Customer Gateway 1
In the Name field, enter Customer-Gateway1.
IP Address: Enter the VPN IP address of the on-premises gateway device: 192.168.10.136.
ASN: The BGP ASN of the customer gateway. For example, 65530.
Customer Gateway 2
For Name, enter Customer-Gateway2.
IP Address: Enter the IP address of the on-premises gateway device, such as 192.168.40.159.
ASN: The BGP ASN of the customer gateway. Enter 65530.
Create an IPsec-VPN connection.
In the left-side navigation pane, choose .
On the IPsec Connections page, click Bind CEN.
On the Create IPsec-VPN Connection (CEN) page, set the following parameters for the IPsec-VPN connection, and then click OK.
The following table describes only the key parameters. Use the default values for the other parameters. For more information, see Create and manage IPsec-VPN connections in dual-tunnel mode.
Configuration item
IPsec-VPN connection
Name
Enter IPsec-Connection.
Region
Select the region of the transit router to attach.
The IPsec-VPN connection is created in the same region as the transit router.
Gateway Type
Select Private.
Attach Cloud Enterprise Network
Select Current-Account Attachment.
CEN Instance ID
Select the created CEN instance.
The system displays the instance ID and CIDR block of the transit router that is created by the CEN instance in the current region. The IPsec-VPN connection will be attached to the transit router.
Routing Mode
This topic uses Destination Routing Mode to control traffic routing.
Enable BGP
Enable the BGP feature.
Local ASN
Enter the BGP AS number of the IPsec-VPN connection, 65534.
Tunnel 1
Customer Gateway
Associate Customer-Gateway1.
Pre-Shared Key
fddsFF111****.
ImportantThe IPsec-VPN connection and the peer gateway device must use the same pre-shared key. Otherwise, the system cannot establish an IPsec-VPN connection.
Encryption Configuration
Keep the default values for all parameters except the following.
Set the DH Group (Perfect Forward Secrecy) parameter in the IKE Configurations section to group14.
Set the DH Group (Perfect Forward Secrecy) parameter in the IPsec Configuration section to group14.
NoteYou need to select encryption parameters based on the on-premises gateway device to ensure that the encryption configurations for the IPsec connection are the same as those for the on-premises gateway device.
BGP Configuration
Tunnel CIDR Block: Enter 169.254.10.0/30.
Local BGP IP address: Enter 169.254.10.1.
Tunnel 2
Customer Gateway
Associate Customer-Gateway2.
Pre-Shared Key
fddsFF222****.
Encryption Configuration
Keep the default values for all parameters except the following.
Set the DH Group (Perfect Forward Secrecy) parameter in the IKE Configurations section to group14.
Set the DH Group (Perfect Forward Secrecy) parameter in the IPsec Configurations section to group14.
NoteYou need to select encryption parameters based on the on-premises gateway device to ensure that the encryption configurations for the IPsec connection are the same as those for the on-premises gateway device.
BGP Configuration
Tunnel CIDR Block: Enter 169.254.20.0/30.
Local BGP IP address: Enter 169.254.20.1.
Advanced Configuration (including route table association and route forwarding)
Use the default route configuration, which enables all advanced configuration options.
After the IPsec-VPN connection is created, you can view its gateway IP addresses on the IPsec-VPN connection details page. The IPsec-VPN connection will use these two gateway IP addresses to establish a private IPsec-VPN connection with the on-premises gateway device.
On the IPsec Connections page, find the IPsec-VPN connection that you create and click Generate Peer Configuration in the Actions column.
The configurations of the IPsec peer refer to the VPN configurations that you need to add when you create the IPsec-VPN connection. In this example, you need to add the VPN configurations to the gateway device of the data center.
In the IPsec Connection Configuration dialog box, copy and save the configurations to an on-premises machine. The configurations are required when you configure the gateway device of the data center.
Configure the on-premises gateway device.
After you create an IPsec-VPN connection, you need to add VPN configurations to your on-premises gateway device to establish a private IPsec-VPN connection between the on-premises gateway device and Alibaba Cloud.
After you complete the configuration, the customer gateway device can establish private IPsec-VPN connections with Alibaba Cloud, but the BGP peers are not yet established. You can check the connection status on the details page of the IPsec-VPN connection. If the private IPsec-VPN connections are not established, you must troubleshoot the issue. For more information, see Self-service diagnostics for IPsec-VPN connections.
Substep 2: Configure routes
After the private IPsec-VPN connection is created, traffic between the data center and the VPC is still transmitted over the Express Connect circuit and is not yet encrypted. You need to add relevant routes to direct traffic between the data center and the VPC through the private IPsec-VPN connection.
Add BGP routing configurations on the on-premises gateway device.
#Configure prefix-list and route-map. prefix-list VPN permit 192.168.10.0/16 prefix-list VPN permit 192.168.20.0/16 prefix-list VPN permit 192.168.40.0/16 route-map VPN permit 10 match ip address prefix-list VPN #Configure BGP to create a BGP peer between the customer gateway device and the IPsec-VPN connection. router bgp 65530 address-family ipv4 unicast neighbor 169.254.10.1 remote-as 65534 #Specify the BGP peer, which is the IP address of Tunnel 1 on the Alibaba Cloud side. neighbor 169.254.10.1 activate #Activate the BGP peer. neighbor 169.254.10.1 route-map VPN out #Advertise only specific routes to Tunnel 1. neighbor 169.254.20.1 remote-as 65534 #Specify the BGP peer, which is the IP address of Tunnel 2 on the Alibaba Cloud side. neighbor 169.254.20.1 activate #Activate the BGP peer. neighbor 169.254.20.1 route-map VPN out #Advertise only specific routes to Tunnel 2. maximum-paths 5 #Increase the number of ECMP route entries. network 192.168.10.0 mask 255.255.255.0 #Advertise the data center CIDR block, which must be more specific than the data center CIDR block advertised to the VBR. network 192.168.20.0 mask 255.255.255.0 network 192.168.40.0 mask 255.255.255.0 exit-address-familyImportantWhen you advertise the data center CIDR block to the private IPsec-VPN connection, make sure that the CIDR block is more specific than the one advertised to the VBR instance. This ensures that the route to the data center learned by the transit router through the private IPsec-VPN connection is preferred.
Add custom routes for the transit router.
After you add the preceding routes, the private IPsec-VPN connections will be interrupted. In this case, you need to add specific routes pointing to the VPN IP addresses of the customer gateway device to the route table of the transit route, and set the next hops to the VBR to recreate the private IPsec-VPN connections.
On the Route Table tab of the CEN console, click the Route Entry tab and click Add Route Entry.
In the Add Route Entry dialog box, configure the parameters and click OK. The following table describes the parameters.
Parameter
CIDR block 1
CIDR block 2
Destination CIDR
Enter one of the VPN IP addresses of the customer gateway device, which is 192.168.10.136/32.
Enter the other VPN IP address of the customer gateway device, which is 192.168.40.159/32.
Whether it is a black hole route
Select No.
Next Hop Connection
Select VBR-Attachment.
Add the first routing policy to the route table of the transit router.
The data center learns the VPC CIDR block from both the VBR instance and the private IPsec-VPN connection. To ensure that traffic from the on-premises environment to the VPC is preferentially routed through the private IPsec-VPN connection, you must configure a routing policy in the transit router. This policy lowers the priority of the route to the VPC that is advertised from the VBR instance to the data center.
Log on to the CEN console.
On the Instances page, find the target CEN instance and click its instance ID.
On the CEN instance details page, click the transit router ID for the China (Hangzhou) region.
On the Transit Router details page, click the Transit Router Route Table tab. Then, click Routing Policy.
On the Route Maps tab, click Add Route Map, set the following parameters, and click OK:
The following table describes only the key parameters. Use the default values for the other parameters. For more information, see Route map overview.
Configuration item
Routing policy
Policy Priority
Enter 30.
Associated Route Table
Keep the default value.
Direction
Select Egress Regional Gateway.
Match Conditions
Destination Instance IDs: The instance ID of the VBR.
Route Prefix: Enter 172.16.10.0/24 and 172.16.20.0/24, and select Exact Match.
Policy Action
Select Allow.
Add Policy Value
In this example, Prepend AS Path is selected and 65525, 65526, and 65527 are specified. This reduces the priority of the VPC CIDR block that the VBR instance advertises to the data center.
Add a second routing policy to the route table of the transit router. This policy denies the private IPsec-VPN connection from advertising routes to the VPN IP addresses of the on-premises gateway device back to the data center, which prevents routing loops.
Set Policy Priority to 40.
Associated Route Table: For this parameter, use the default value.
For Direction, select Egress Regional Gateway.
Match conditions:
Destination Instance IDs: The instance IDs of the destination IPsec-VPN connections.
Route Prefix: Select Exact Match and enter the VPN IP addresses of the on-premises gateway device: 192.168.10.136/32 and 192.168.40.159/32.
Set Policy Action to Reject.
Step c: Verify the encryption setting
After you complete the configuration, if you can view the data transfer details on the details page of the IPsec-VPN connection, traffic is encrypted.
Connect to ECS 1 in the VPC. Run the ping command to ping a client in the CIDR block of the data center.
ping <IP address of a client in the data center> -s 1000 -c 10000-s 1000: Send 1,000 bytes.-c 10000: Continuously send 10,000 requests.
Log on to the VPN Gateway console.
In the top navigation bar, select the China (Hangzhou) region.
In the left-side navigation pane, choose .
On the IPsec Connections page, find the IPsec-VPN connection that you created and click its ID.
Go to the details page of the IPsec-VPN connection to view the details of data transfer.
