Configure routes for ENIs - Elastic Compute Service - Alibaba Cloud Documentation Center

When data is transmitted within a virtual private cloud (VPC) and between the VPC and other networks, a route table is used to route packets. Correct routing configurations help ensure that elastic network interfaces (ENIs) send and receive data as expected. When you bind ENIs to Elastic Compute Service (ECS) instances, default routes are automatically configured for the ENIs. This topic describes how to configure route tables and rules to ensure that existing routes meet your business requirements.

Configure policy-based routes for ENIs

Policy-based routing is a networking method that makes routing decisions based on pre-defined rules, not just the default routes in route tables. You can configure route tables and rules to implement different routing policies and forward traffic through specified network interfaces. This way, you can control and manage network traffic in a finer-grained manner.

Scenarios

Communication in a multi-network interface environment: If you use secondary ENIs together with EIPs or NAT gateways, outbound traffic is preferentially sent by primary ENIs because the default routes of primary ENIs have higher priorities than the default routes of secondary ENIs. As a result, after ECS instances receive data traffic on secondary ENIs, the instances may send reply data traffic from primary ENIs, which causes communication issues. In this case, you can configure policy-based routes to route data traffic in and out through the same ENIs based on the source in-source out principle.
Traffic load-balancing: In a multi-network interface environment, each network interface may connect to a different network or serve a different purpose. As a result, default routes may not suffice. In this case, you can configure policy-based routes for network interfaces. For example, you can configure policy-based routes to distribute and balance outbound traffic across different network interfaces based on rules.
Access control: Define routing rules based on source addresses, destination addresses, or other parameters to restrict specific types of traffic to specific interfaces or paths to achieve network isolation.

Prerequisites

ENIs are bound to an ECS instance.
For more information, see the Bind an ENI to an ECS instance section of the "Create and manage ENIs" topic.
The ENIs are in effect in the operating system of the ECS instance.
For more information, see the Configure ENIs to take effect in an ECS instance section of the "Create and manage ENIs" topic.

Procedure

In this section, a secondary ENI named eth1 is used. Replace the information of the ENI, such as the network interface identifiers and IP addresses, with actual values.

Linu instance

Run the following commands to create a route table for an ENI and add a route rule to the route table:
```
ip -4 route add default via <Gateway of eth1> dev eth1 table 1001
ip -4 rule add from <IP address of eth1> lookup 1001
```
For example, run the following commands to create a route table named 1001 for the eth1 secondary ENI and add a route rule for packets originating from 172.16.20.193 to the route table. This ensures that packets originating from 172.16.20.193 are sent through eth1.
```
ip -4 route add default via 172.16.20.253 dev eth1 table 1001
ip -4 rule add from 172.16.20.193 lookup 1001
```
Configure the ECS instance to automatically update routing information on instance startup and allow the configuration to permanently take effect.
After you configure a default route for eth1, add the relevant command to the configuration file of the instance to automatically update routing information on instance startup. Otherwise, the default route configuration becomes invalid after the instance is restarted. If you perform a test by using the default route, skip this step.
1. Run the following command to open the /etc/rc.local file:
```
vim /etc/rc.local
```
2. Press the i key to enter Insert mode, add the command that you ran in the previous step to the file, and then press the Esc key to exit Insert mode. Enter :wq and press the Enter key to save and close the file.
  Note
  In the command, replace relevant information, such as the network interface identifier and gateway address, with actual values.
3. Run the following command to grant execute permissions on the /etc/rc.local file:
```
sudo chmod +x /etc/rc.local
```
Run the following commands to check whether the route table is created and the route rule is added to the route table:
```
ip route list table 1001 && \
ip rule list
```
The following command output indicates that the route table is created and the route rule is added to the route table.

Windows instance

Run the following command to create a policy-based route for an ENI:
```
route add -p <Destination network> mask <Subnet mask> <Gateway> if <Interface index> metric <Route priority>
```
In this example, an ENI named Ethernet 2 is used. Run the following command to configure a route that forwards all packets originating from 172.16.12.76 through the gateway at 172.16.12.253.
```
route add -p 0.0.0.0 mask 0.0.0.0 172.16.12.253 if 6 metric 1
```
Take note of the following parameters:
- -p: makes the route permanent. The permanent route remains valid after a system restart. By default, the -p parameter is not specified, which means that the route is temporary.
- Destination network: In most cases, a value of 0.0.0.0 indicates the default route, which is used when no specific routes are matched.
- Subnet mask and gateway: The subnet mask and gateway of the ENI. Run the ipconfig command to view the subnet mask and gateway of an ENI, as shown in the following figure.
- Interface index: the index of the ENI. Run the netsh interface ipv4 show interfaces command to view the index of an ENI, as shown in the following figure.
- Route priority: the priority of the route. Specify a value in the metric <n> format. A smaller value indicates a higher priority.
Run the route print command to check whether the created route appears in the route list.

Example

This section describes how to configure policy-based routes to ensure that an ECS instance receives packets through a secondary ENI named eht1 and sends reply packets through the primary ENI named eth0. In this example, an ECS instance that runs Alibaba Cloud Linux 3.2 is used. If you have strict requirements for data traffic directions, such as when a source IP address whitelist in your security policies includes only the IP addresses of specific network interfaces, incorrect traffic path binding may cause legitimate requests to be rejected. This issue occurs due to a mismatch between the source IP addresses and the network interfaces and results in network connectivity failures. To resolve the issue, you can configure policy-based routes.

Prepare an environment.
1. Create an ECS instance.
  For information about how to create an ECS instance, see Create an instance on the Custom Launch tab.
2. Bind a secondary ENI to the ECS instance.
  For information about how to bind a secondary ENI to an ECS instance, see Bind a secondary ENI.
3. Apply for an elastic IP address (EIP) and associate the EIP with the secondary ENI named eht1 that is bound to the ECS instance in NAT mode.
  For more information, see Associate an EIP with a secondary ENI.
4. Prepare a test client.
  You can use a different ECS instance that has Internet access or an on-premises computer as a test client.
5. Add an inbound rule that allows access from the public IP address of the test client to a security group of the ECS instance that you created in Step a. Then, you can run the ping <EIP> command on the test client to ping the EIP that you associated in Step c and access the ECS instance.
  For more information, see the Case 4: Allow only traffic of specific protocols to access ECS instances section of the "Guidelines for using security groups and use cases" topic.
Run the following command on the test client to send packets to the ECS instance:
```
ping 47.xx.xx.109
```
In the preceding command, replace the IP address with the EIP that is associated with the eth1 secondary ENI.
Monitor Internet Control Message Protocol (ICMP) packets on the primary ENI named eth0 and the eth1 secondary ENI on the ECS instance.
- Run the following command to capture ICMP packets on the eth0 primary ENI:
```
tcpdump -i eth0 icmp
```
- Open a new window and run the following command to capture ICMP packets on the eth1 secondary ENI:
```
tcpdump -i eth1 icmp
```
View the results.
Policy-based route not configured
The following figure shows that packets enter through eth1 and corresponding reply packets exit through eth0. The ECS instance receives requests through eth1 and sends responses through eth0.
The default route of eth0 has a priority of 100, which is higher than the priority of the default route of eth1. As a result, packets are sent from eth0.
Policy-based route configured
1. Configure policy-based routes for the ECS instance. For more information, see the Configure policy-based routes for ENIs section of this topic.
```
ip -4 route add default via 172.16.20.253 dev eth1 table 1001
ip -4 rule add from 172.16.20.177 lookup 1001
```
2. Run the ping <EIP> command on the test client.
3. Monitor ICMP packets on the ENIs of the ECS instance.
  The following figure shows that packets enter through eth1 and corresponding reply packets exit through eth1. The ECS instance receives requests and sends responses through eth1 based on the source in-source-out principle.

Configure default routes for ENIs

When you bind ENIs to ECS instances, default routes are automatically configured for the ENIs. In specific operating system versions earlier than Ubuntu 18, such as Ubuntu 16, default routes may not be automatically configured for secondary ENIs. If default routes are not configured, network connectivity issues may occur when you use ENIs for external communication. To configure a default route for an ENI, perform the following steps.

In this example, Ubuntu16 and the eth1 secondary ENI are used.

Run the following command to view ENI information:
```
ip a
```
The preceding command output indicates that eth1 is in effect in the operating system of the ECS instance.
Run the following command to view route information:
```
route -n
```
The following command output indicates that eth1 has a route only for internal communication and does not have outbound routes.
Example exception
The preceding scenario may cause an exception when you specify eth1 for communication. For example, network connectivity may be lost when you associate an EIP with eth1 and configure the ECS instance to communicate with the Internet by using the EIP, as shown in the following figure.
Run the following command to configure the default route for eth1:
```
ip -4 route add default via 172.16.20.253 dev eth1 metric 200
```
- -4: applies only to IPv4 addresses.
- 172.16.20.253: the gateway address of eth1.
- metric 200: sets the metric value (priority) of the route to 200. A smaller metric value indicates a higher priority. The route with the smallest metric value is used when multiple routes to the same destination exist.
Configure the ECS instance to automatically update routing information on instance startup and allow the configuration to permanently take effect.
After you configure a default route for eth1, add the relevant command to the configuration file of the instance to automatically update routing information on instance startup. Otherwise, the default route configuration becomes invalid after the instance is restarted. If you perform a test by using the default route, skip this step.
1. Run the following command to open the /etc/rc.local file:
```
vim /etc/rc.local
```
2. Press the i key to enter Insert mode, add the command that you ran in the previous step to the file, and then press the Esc key to exit Insert mode. Enter :wq and press the Enter key to save and close the file.
  Note
  In the command, replace relevant information, such as the network interface identifier and gateway address, with actual values.
3. Run the following command to grant execute permissions on the /etc/rc.local file:
```
sudo chmod +x /etc/rc.local
```
Run the following command to view the route added for eth1:
```
route -n
```
Verify that the exception is resolved
Configure eth1 for internal communication. The following figure shows that eth1 can communicate as expected.