Preserve client IP addresses - Global Accelerator - Alibaba Cloud Documentation Center

To obtain the original IP address of a client when using Global Accelerator (GA) to accelerate application services, you can enable GA's feature to preserve client IP addresses. For Layer 7 listeners, the client's original IP address can be retrieved from the X-Forwarded-For field in the HTTP request header. For Layer 4 listeners, the retrieval method depends on the backend service type; it either supports automatic retrieval or requires the use of the Proxy Protocol.

Note

Only smart routing type listeners support preserving client IP addresses. Custom routing type listeners do not support this feature by default. To preserve client IP addresses, please contact your account manager.

HTTP or HTTPS listeners

Method to obtain

Retrieve the client's original IP address using the Preserve Client IP Address feature, which is enabled by default.

GA stores the client's original IP in the X-Forwarded-For field of the HTTP request header. Configure the backend server to retrieve it. The first IP listed is the client's original IP.

X-Forwarded-For: <client original IP, proxy server 1-IP, proxy server 2-IP,...>

Procedure

This section uses Alibaba Cloud Linux 3 and Nginx 1.20.1 as examples. Adjust the configurations according to your environment.

Run the command nginx -V | grep http_realip_module on the NGINX server to check whether the http_realip_module module is installed on the NGINX server. NGINX servers use the http_realip_module module to parse the value of X-Forwarded-For.

If --with-http_realip_module is included in the output, it indicates that http_realip_module is installed on the NGINX server and you can proceed to the next step.

Sample output which shows that http_realip_module is installed

nginx version: nginx/1.20.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
built with OpenSSL 1.1.1k  FIPS 25 Mar 2021
TLS SNI support enabled
configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-compat --with-debug --with-file-aio --with-google_perftools_module --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'

Note

NGINX 1.0.4, which was released in 2011, and later support the http_realip_module module. If your NGINX version is earlier than 1.0.4, we recommend that you back up data and upgrade to a later version.
If the http_realip_module module is not installed on the NGINX server, compile and install NGINX again, and then install http_realip_module. We recommend that you use package managers such as YUM to install and manage NGINX. Packet managers simplify the process.

Modify and save the NGINX configuration file. The following code block shows an example. Run the nginx -t command to query the path of the configuration file, which is /etc/nginx/nginx.conf by default. The path may vary based on the environment that you use.

http {
  # Set the variable $http_x_forwarded_for, which is used to record the value of X-Forwarded-For.
  log_format  main  '$remote_addr- $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
  
  # ...
}

Run the sudo nginx -s reload command to reload the NGINX configuration file.
Retrieve the client's original IP.

Check the NGINX log to verify if the client's original IP address has been successfully retrieved.

Use the tail -n ＜specified number of log entries＞＜log path＞ command. In the log information, look for the $http_x_forwarded_for variable. The first IP address is the client's source IP.

In this example, the default NGINX log file path is /var/log/nginx/access.log.

Note
If you have enabled health checks for Global Accelerator, consider returning multiple logs for viewing or temporarily increasing the health check interval to prevent log information containing the client's source IP from being overwritten by health check log information.

TCP listeners

Method to obtain

Retrieve the client's original IP address using the Preserve Client IP Address feature. After enabling this feature, select a method to obtain the client's original IP.

The supported methods vary depending on the backend service type.

Automatic retrieval: The client's source IP is passed through directly. The backend server receives the original IP address.
Proxy Protocol: The backend server must adapt to the Proxy Protocol to retrieve the client's original IP address.

What is the Proxy Protocol?

The Proxy Protocol is an Internet protocol that conveys original connection information between proxy servers and backend servers.

Typically, a proxy server overwrites the request header containing the source client IP address and forwards the request to backend servers. The source client IP address and port are replaced by those of the proxy server. Consequently, backend servers lose the original connection information.

The Proxy Protocol allows proxy servers to encapsulate original connection information in a request header, which is then passed to backend servers. Backend servers can obtain the original connection information by parsing the encapsulated request header. This information includes the source IP address, source port, and transmission protocol.

The Proxy Protocol ensures backend servers can access original connection information for detailed logging, access control, and traffic monitoring.
Important
- The Proxy Protocol is effective only when enabled on both the proxy server and backend servers. If a backend server cannot parse Proxy Protocol headers but the protocol is enabled, the backend server may experience parsing errors, affecting service availability.
- GA listeners support carrying original connection information (source IP, destination IP, source port, destination port, etc.) through the Proxy Protocol and adding it to the TCP data header without discarding or overwriting any original data.
- GA only supports Proxy Protocol v1, which is limited to the TCP protocol. For more information, see The PROXY protocol.

Backend service deployed on Alibaba Cloud

Backend service type		Support for obtaining client source IP	Corresponding method to obtain
ECS		Supported However, it must be a virtual private cloud (VPC) type, and the security group where the ECS is located must allow all client source IPs.	Automatic retrieval Proxy Protocol
ALB		Not supported	N/A
NLB		Supported	Automatic retrieval (only supported when the backend server type of the NLB instance is ECS or ENI) Proxy Protocol
CLB		Supported However, the security group where the backend server of the CLB is located must allow all client source IPs. In the following scenarios, the backend servers cannot obtain client IP addresses: The backend servers of the CLB instance are ECS instances that are deployed in classic networks. The HTTP or HTTPS protocol is used for the listeners of the CLB instance.	Automatic retrieval (only supported when the backend server type of the CLB instance is ECS or ENI) Proxy Protocol
OSS		Not supported	N/A
ENI		Supported However, the security group where the ENI is located must allow all client source IPs.	Automatic retrieval Proxy Protocol
Custom Private Network IP		Supported	Proxy Protocol
Alibaba Cloud Public IP Warning When the endpoint is an Alibaba Cloud public IP, if the public IP is unbound from the original instance and re-bound to another instance, the ability to preserve client IP addresses may fail, causing traffic interruption. To restore the ability of the endpoint to preserve client IP addresses, you can delete and rebuild the corresponding endpoint, or contact your account manager for assistance.	Bind to a VPC type ECS instance	Supported	Automatic retrieval Proxy Protocol
	Bind to an NLB instance	Supported	Automatic retrieval (only supported when the backend server type of the NLB instance is ECS or ENI) Proxy Protocol
	Bind to an ALB instance	Not supported	N/A
	Bind to a private network CLB instance	Supported	Automatic retrieval (only supported when the backend server type of the CLB instance is ECS or ENI) Proxy Protocol
	Bind to an ENI	Supported	Automatic retrieval Proxy Protocol
	Bind to a public NAT instance	Supported	Proxy Protocol

Backend service deployed outside Alibaba Cloud

Backend service type	Support for obtaining client source IP	Corresponding method to obtain
Custom Public IP	Supported	Proxy Protocol
Custom Domain Name	Supported	Proxy Protocol

Note

In scenarios where clients use IPv6 addresses to access backend IPv4 services, the Proxy Protocol must be used to obtain the client's original IP address.

Procedure

This section uses Alibaba Cloud Linux 3 and Nginx 1.20.1 as examples. Adjust the configurations according to your environment.

Automatic retrieval

Enable the Preserve Client IP Address feature.
1. Log on to the Global Accelerator console.
2. On the Instance List page, find the target Global Accelerator instance and click Configuration Listening in the Actions column.
3. On the Listening tab, locate the desired TCP listener and click Edit Listener in the Actions column.
4. On the Configure Listening And Protocol configuration wizard page, click Next.
5. On the Configure Endpoint configuration wizard page, select Preserve in the Preserve Client IP Address list, and then click Next.
  
  When the backend service is deployed on Alibaba Cloud, the Method To Obtain Client Original IP is automatically selected as Automatic Retrieval.
6. On the Configuration Review page, confirm that there are no errors, and click Submit.

Check the NGINX service configuration file.

Use the nginx -t command to verify the configuration file path. The default path is typically /etc/nginx/nginx.conf. Adjust the configurations according to your environment.

Note

If the backend service only needs to handle HTTP and HTTPS traffic, ensure that the NGINX server supports the http{} module. If the backend service also needs to handle TCP traffic, confirm that the NGINX server supports the stream{} module.

Since NGINX 1.9.0, the stream{} module has been available but not installed by default. You may need to compile and install it yourself to support the stream{} module.

# Handle HTTP traffic
http {
  # Default configurations
  log_format  main  '$remote_addr- $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
  access_log  /var/log/nginx/access.log  main;
  
  # For example, listen on port 80
  server {
        listen 80;
        #...
       }
}

# Handle TCP traffic (add as needed)
stream {
    log_format main_stream '$remote_addr - [$time_local] '
                        '"$protocol" $status $bytes_sent $bytes_received '
                        '"$session_time"';

    access_log  /var/log/nginx/stream_access.log  main_stream;
# For example, listen on port 4000
    server {
        listen 4000;
        #...
    }
}

Retrieve the client's source IP.
- Method 1: Check the NGINX log to see if the client's original IP address has been successfully retrieved.
  
  Use the tail -n ＜specified number of log entries＞＜log path＞ command. In the log information, look for the first IP address corresponding to the $remote_addr variable, which is the client's source IP.
  - For HTTP traffic:
    
    In this example, the log file path is /var/log/nginx/access.log.
  - For TCP traffic:
    
    In this example, the log file path is /var/log/nginx/stream_access.log.
- Method 2: Capture packets to verify if the client's original IP address has been successfully retrieved.
  
  Use the tcpdump tcp port [listening port] -n -X -s 0 command to view the client's original IP address from the captured packets.

Obtain through proxy protocol

Enable the Preserve Client IP Address feature.
1. Log on to the Global Accelerator console.
2. On the Instance List page, find the target Global Accelerator instance and click Configuration Listening in the Actions column.
3. On the Listening tab, locate the desired TCP listener and click Edit Listener in the Actions column.
4. On the Configure Listening And Protocol configuration wizard page, click Next.
5. On the Configure Endpoint configuration wizard page, select Preserve in the Preserve Client IP Address tab, and then click Next.
  
  When the backend service is deployed outside Alibaba Cloud, the Method To Obtain Client Original IP is automatically selected as ProxyProtocol.
6. On the Configuration Review page, confirm that there are no errors, and click Submit.

Modify and save the NGINX configuration file as shown in the example below.

Use the nginx -t command to check the configuration file path, typically /etc/nginx/nginx.conf. Adjust the configurations according to your environment.

Note

Since NGINX 1.9.0, the stream{} module has been available but not installed by default. You may need to compile and install it yourself to support the stream{} module.

# Handle HTTP traffic
http {
  # Ensure to add the variable $proxy_protocol_addr that saves the client source IP in the log format of the http{} module
  log_format  main  '$proxy_protocol_addr - $remote_addr- $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';     
                                    
  access_log  /var/log/nginx/access.log  main;
  
  # For example, listen on port 80, enable parsing proxy_protocol
  server {
    listen 80   proxy_protocol;
    #...
  }
}

# Handle TCP traffic (add as needed)
stream {
    # Ensure to add the variable $proxy_protocol_addr that saves the client source IP in the log format of the stream{} module
    log_format  main_stream  '$proxy_protocol_addr - $remote_addr - $protocol [$time_local] '
                              '$status $bytes_sent $bytes_received $session_time';
                              
    access_log /var/log/nginx/stream_access.log main_stream; 

   # For example, listen on port 12345, enable parsing proxy_protocol
    server {
        listen 12345 proxy_protocol;
        #...
    }
}

Reload the NGINX configuration file by executing the sudo nginx -s reload command.
Retrieve the client's source IP.

Check the NGINX log to verify if the client's original IP address has been successfully retrieved.

Use the tail -n ＜specified number of log entries＞＜log path＞ command. In the log information, look for the IP address corresponding to the $proxy_protocol_addr variable, which is the client's source IP.
- For HTTP traffic:
  
  This topic specifies the log file path as: /var/log/nginx/access.log.
- For TCP traffic:
  
  In this example, the log file path is /var/log/nginx/stream_access.log.

UDP listeners

Method to obtain

Use the Preserve Client IP Address feature to obtain the client's original IP address for UDP listeners. By default, only the Automatic Retrieval method is supported, which directly passes the client's source IP to the backend server.

Note the following limitations when enabling Preserve Client IP Address for UDP listeners:

Only pay-as-you-go GA instances are supported.

If your pay-as-you-go GA instance does not support enabling Preserve Client IP Address for UDP listeners due to the instance version, please contact your account manager for an upgrade.
Only supports Backend Service Type as NLB, ECS, ENI. Among them, the NLB type's Preserve Client IP Address feature is not open by default. If you need to use it, please contact your account manager.
Only IPv4 client source IPs can be obtained; IPv6 client source IPs are not supported.

Procedure

This section uses Alibaba Cloud Linux 3 and Nginx 1.20.1 as examples. Adjust the configurations according to your environment.

Enable the Preserve Client IP Address feature.
1. Log on to the Global Accelerator console.
2. On the Instance List page, find the target Global Accelerator instance and click Configuration Listening in the Actions column.
3. On the Listening tab, find the target UDP listener and click Edit Listener in the Actions column.
4. On the Configure Listening And Protocol configuration wizard page, click Next.
5. On the Configure Endpoint configuration wizard page, select Preserve in the Preserve Client IP Address tab, and then click Next.
  
  At this point, the Method To Obtain Client Original IP is automatically set to Automatic Retrieval.
6. On the Configuration Review page, confirm that there are no errors, and click Submit.
Check the NGINX service configuration file.

Use the nginx -t command to verify the configuration file path. The default path is typically /etc/nginx/nginx.conf. Adjust the configurations according to your environment.

Note
To handle UDP traffic, ensure that the NGINX server supports the stream{} module.

Since NGINX 1.9.0, the stream{} module has been available but not installed by default. You may need to compile and install it yourself to support the stream{} module.
```
#...

# Handle UDP traffic (ensure to include this module)
stream {
    log_format main_stream '$remote_addr - [$time_local] '
                        '"$protocol" $status $bytes_sent $bytes_received '
                        '"$session_time"';

    access_log  /var/log/nginx/stream_access.log  main_stream;
# For example, listen on UDP port 4000
    server {
        listen 4000 udp;
        #...
    }
}
```
Obtain the client's source IP.
- Method 1: Check the NGINX log to see if the client's original IP address has been successfully retrieved.
  
  Use the tail -n ＜specified number of log entries＞＜log path＞ command. In the log information, look for the first IP address corresponding to the $remote_addr variable, which is the client's source IP.
  
  In this example, the log file path is /var/log/nginx/stream_access.log.
- Method 2: Capture packets to verify if the client's original IP address has been successfully retrieved.
  
  Use the tcpdump udp port [listening port] -n -X -s 0 command to view the client's original IP address from the captured packets.

FAQ

What are the possible reasons for the preserve client IP address feature not taking effect after it is enabled?

If you use an HTTP or HTTPS listener and the backend server cannot record or process the X-Forwarded-For header, client IP address preservation does not take effect.
If you use a TCP listener:
- If the backend service is deployed on Alibaba Cloud but the backend servers are not deployed in a virtual private cloud (VPC) or the elastic IP address (EIP) is not associated with an instance of the VPC type, the backend servers cannot preserve client IP addresses.
- If the backend service is deployed outside Alibaba Cloud, the backend servers must support the Proxy Protocol so that they can preserve client IP addresses.
  This is because GA uses Proxy Protocol to preserve client IP addresses after you enable client IP address preservation. Therefore, you must make sure that the backend servers support Proxy Protocol. Otherwise, the backend servers cannot parse the Proxy Protocol header.
- If the client uses an IPv6 address to access an IPv4 backend service, the backend servers must support Proxy Protocol. Otherwise, the backend servers cannot preserve client IP addresses.
If you use a UDP listener and the client uses an IPv6 address to access an IPv4 backend service, client IP address preservation does not take effect.

How do I disable client IP address preservation?

For HTTP listeners and HTTPS listeners, client IP address preservation is automatically enabled and cannot be disabled.

For TCP and UDP listeners, you can disable client IP address preservation in the GA console. Perform the following steps:

On the Instances page, find the GA instance that you want to manage and click Configure Listeners in the Actions column.
On the Listeners tab, find the listener that you want to manage and click Mod Lsnr in the Actions column.
In the Configure Listener & Protocol step, click Next.
In the Configure Endpoint Group step, set Preserve Client IP to Do Not Preserve and click Next.
In the Configuration Review step, confirm the information and click Submit.

How to obtain the client source IP when Global Accelerator IPv6 is transformed to IPv4?

HTTP and HTTPS listeners obtain the client IP addresses of IPv6 and IPv4 clients by using the X-Forwarded-For HTTP header.
TCP listeners obtain the client IP addresses of IPv6 clients by using Proxy Protocol. This method requires that the backend servers support Proxy Protocol. Otherwise, access may fail.
UDP listeners cannot obtain the client IP addresses of IPv6 clients.

References

Console operation documentation

You can also retrieve client source IPs using the access log feature of GA. By setting up access logs for the designated endpoint group, these logs are sent to the Logstore in SLS for the region where the endpoint group resides. On the Original Log tab of the corresponding Logstore page, you can view the access request details for the client source IP in the client_ip field.
For more information on configuration and precautions regarding the backend services of the endpoint group, see Endpoint Group and Endpoint and Add and Manage Endpoint Groups of Smart Routing Type Listeners.

API documentation

When calling the APIs below, you can specify the method to capture the client's original IP using the parameters EnableClientIPPreservation (automatic retrieval method) and EnableProxyProtocol (ProxyProtocol method).