Associate multiple EIPs with an ECS instance in NAT mode - Elastic IP Address

To host different services on a single Elastic Compute Service (ECS) instance using multiple Elastic IP Addresses (EIPs), associate the EIPs with an elastic network interface (ENI) on the instance in NAT mode.

You can associate multiple EIPs with both a primary ENI and a secondary ENI. This topic uses a secondary ENI as an example.

How it works

Associate multiple EIPs: Attach a secondary ENI with multiple private IP addresses to an ECS instance. Then, associate multiple EIPs with these private IP addresses one-to-one in NAT mode.
Configure policy-based routing to ensure symmetric traffic paths:
- Inbound traffic: When an external request accesses an EIP, the system automatically forwards the traffic to the private IP address associated with that EIP.
- Outbound traffic: Configure policy-based routing on the ECS instance to ensure that the instance returns response traffic from the correct EIP. Policy-based routing determines the next hop and egress device (the secondary ENI) based on the source IP address of the data packet (the private IP address). This configuration ensures symmetric routing in a multi-NIC environment and prevents routing conflicts.

Usage notes

The number of ENIs you can attach and secondary private IP addresses you can assign depends on the instance family.
For example, an ecs.c6.large instance supports a maximum of 2 attached ENIs (one primary and one secondary), and each ENI supports up to 6 private IP addresses. On both the primary and secondary ENIs, you can associate one EIP with the primary private IP address and five EIPs with the secondary private IP addresses.
EIPs associated in NAT mode do not support protocols that require a NAT Application Level Gateway (ALG).
- EIPs can be associated with both primary and secondary ENIs.
- The number of EIPs you can associate depends on the number of private IP addresses on the ENI, with a one-to-one mapping between them.
To allow the instance to access the Internet, add a route to the IPv4 gateway if the instance's virtual private cloud (VPC) uses an IPv4 gateway for centralized Internet access.

Procedure

Step 1: Associate multiple EIPs

Attach a secondary ENI with multiple private IP addresses to the ECS instance.
If you already have a secondary ENI, you can bind it directly to the ECS instance.
1. Go to the ECS console - ENIs page and select the instance's region.
2. Click Create ENI.
  Keep the default settings for the other parameters. After creation, click Manage ENI IP Addresses in the Operation column of the target ENI to add, delete, or modify private IP addresses.
  - Select the VPC, vSwitch, and security group of the ECS instance.
  - Primary Private IP Address: Specify an unused IP address in the vSwitch. Alternatively, leave the parameter empty, and the system will randomly assign an available one. This parameter value cannot be changed after creation.
  - Secondary Private IPv4 Address: Select Auto-assign and enter the number of secondary private IP addresses you plan to assign.
3. In the Operation column for the target ENI, click Bind to Instance and select the target ECS instance.
Configure the operating system to recognize the secondary private IP addresses.
In this example, the target ECS instance runs an Alibaba Cloud Linux 3.2 operating system. For information about recognizing IP addresses for other operating systems, see Configure the operating system of the instance to recognize the secondary private IP addresses.
1. Log on to the ECS instance and run the ip a command to view and confirm the ENI information.
  - Network interface identifiers: eth0 (primary ENI), eth1 (secondary ENI).
  - Network interface status: state UP indicates that the ENI is functioning correctly within the instance. If the status is state DOWN, you must configure the Linux OS to recognize the ENI.
2. Use nmcli con to configure the secondary private IP addresses.
  1. To ensure the configuration persists after a restart, disable automatic network initialization by cloud-init by running sudo vim /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg to create a file and add the network: {config: disabled} configuration.
  2. Run nmcli con show to view the connection name for eth1.
  3. Run the following commands to configure the secondary private IPv4 addresses and the default gateway for the secondary ENI.
    You can run route -n to view the default gateway.
```
sudo nmcli con modify "<eth1 connection name>" ipv4.addresses <Secondary private IPv4 address 1>,<Secondary private IPv4 address 2>
sudo nmcli con modify "<eth1 connection name>" ipv4.gateway <default gateway>
```
3. Run sudo nmcli con up "<eth1 connection name>" to activate the modified network connection. A Connection successfully activated message indicates a successful configuration. Run ip a again to view the secondary private IP addresses.
Create multiple EIPs and associate them with the secondary ENI.
1. Go to the EIP buy page.
  Only relevant configuration items are listed here. For detailed selection guidelines, see EIP selection guide.
  - Billing Method: The pay-as-you-go billing model is suitable for fluctuating workloads, while subscription is ideal for long-term, stable services. In this example, select Pay-as-you-go.
  - Region: Must be the same as the region of the ECS instance.
  - Line Type: You can customize this parameter only in some regions.
  - Security Protection: You can customize this parameter only for BGP (Multi-ISP) EIPs that use the pay-as-you-go billing method.
  - IP Address Pool: You can allocate EIPs from an IP address pool if you have one.
  - Quantity: Select the number of EIPs you plan to associate.
2. Individually associate each EIP with a private IP address on the secondary ENI.
  1. Go to the VPC console - Elastic IP Addresses page and select the region where the EIPs are located.
  2. In the Actions column of a target EIP, click Associate with Resource. In the dialog box that appears, select ENI, then choose a secondary private IP address.

Step 2: Configure policy-based routing

Configure policy-based routing on the ECS instance to ensure symmetric traffic paths.

Log on to the ECS instance and add a default route for the secondary ENI eth1 by creating a route table table 1001 and the routing policies for the private IP addresses.

ip -4 route add default via <Default gateway> dev eth1 metric 1001 && \
ip -4 route add default via <Default gateway> dev eth1 table 1001 && \
ip -4 rule add from <Private IPv4 address 1 of the secondary ENI> lookup 1001 && \
ip -4 rule add from <Private IPv4 address 2 of the secondary ENI> lookup 1001 && \
ip -4 rule add from <Private IPv4 address 3 of the secondary ENI> lookup 1001

Run ip route list table 1001 && ip rule list to view the created route table and policy-based routes.
Configure the routes to apply automatically at system startup to ensure the configuration persists through instance restarts.
1. Run vim /etc/rc.local and add the commands for creating the route table and policy-based routes to the file.
2. Run sudo chmod +x /etc/rc.local to make the file executable.

Step 3: Verify the configuration

Verify the outbound IP address

Log on to the ECS instance and run curl --interface <Private IP address of the secondary ENI> https://ifconfig.me. Confirm that the public egress IP address for requests sent from different private IP addresses match the associated EIP.

Verify the inbound and outbound traffic paths

Log on to another test ECS instance that can access the Internet and run ping <EIP associated with the secondary ENI>.
At the same time, log on to the target ECS instance and run tcpdump -i eth1 icmp to capture ICMP packets on eth1.

You can find that packets both enter and exit through eth1, which confirms that the traffic path is symmetric.

Apply in production

Risk prevention: This solution uses a single ECS instance. We recommend using Server Load Balancer (SLB) to ensure high availability (HA) for your services. Note that business interruptions may occur if the instance restarts or network configurations change.
Monitoring and alerting: Set up monitoring and alerting for key metrics of the secondary ENI, such as network traffic, CPU, and memory utilization, to detect anomalies promptly.
Security hardening: Configure security group rules for the secondary ENI and allow only necessary ports and source IP addresses to access your services.

Billing

EIP billing:
- EIP configuration fees (public IP retention fees):
  - Pay-as-you-go EIPs: When an EIP is associated with a secondary ENI, you are charged a configuration fee for the public IP address retention, even if there is no Internet traffic.
  - Subscription EIPs: No configuration fee is charged.
- Internet data transfer fees:
  - Pay-as-you-go EIPs: Fees are charged based on the selected metering method (pay-by-bandwidth or pay-by-data-transfer).
  - Subscription EIPs: Fees are charged based on the selected metering method (pay-by-bandwidth or pay-by-data-transfer).
Other resource billing: Resources such as ECS instances are billed according to their own rules. Secondary ENIs are free of charge.