Global Accelerator:Preserve client IP addresses

Method to obtain client IP addresses

You can use the client IP address preservation feature. By default, this feature is enabled.

GA stores the IP addresses of clients in the X-Forwarded-For field of HTTP request headers. After you configure backend servers, the backend servers can obtain client IP addresses. The first IP address is the real IP address of a client.

X-Forwarded-For: <Client IP, Proxy Server 1-IP, Proxy Server 2-IP, ...>

Procedure

In this example, the Alibaba Cloud Linux 3 operating system and NGINX 1.20.1 are used. Adjust the configurations based on the environment that you use.

1. Run the command nginx -V | grep http_realip_module on the NGINX server to check whether the http_realip_module module is installed on the NGINX server. NGINX servers use the http_realip_module module to parse the value of X-Forwarded-For.

   If --with-http_realip_module is included in the output, it indicates that http_realip_module is installed on the NGINX server and you can proceed to the next step.

   Sample output which shows that http_realip_module is installed

   nginx version: nginx/1.20.1
   built by gcc 4.8.5 20150623 (Red Hat 4.8.5-44) (GCC) 
   built with OpenSSL 1.1.1k  FIPS 25 Mar 2021
   TLS SNI support enabled
   configure arguments: --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --http-client-body-temp-path=/var/lib/nginx/tmp/client_body --http-proxy-temp-path=/var/lib/nginx/tmp/proxy --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/run/nginx.pid --lock-path=/run/lock/subsys/nginx --user=nginx --group=nginx --with-compat --with-debug --with-file-aio --with-google_perftools_module --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_degradation_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_image_filter_module=dynamic --with-http_mp4_module --with-http_perl_module=dynamic --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_xslt_module=dynamic --with-mail=dynamic --with-mail_ssl_module --with-pcre --with-pcre-jit --with-stream=dynamic --with-stream_ssl_module --with-stream_ssl_preread_module --with-threads --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic' --with-ld-opt='-Wl,-z,relro -specs=/usr/lib/rpm/redhat/redhat-hardened-ld -Wl,-E'
   

   Note

   • NGINX 1.0.4, which was released in 2011, and later support the http_realip_module module. If your NGINX version is earlier than 1.0.4, we recommend that you back up data and upgrade to a later version.

   • If the http_realip_module module is not installed on the NGINX server, compile and install NGINX again, and then install http_realip_module. We recommend that you use package managers such as YUM to install and manage NGINX. Packet managers simplify the process.

2. Modify and save the NGINX configuration file. The following code block shows an example. Run the nginx -t command to query the path of the configuration file, which is /etc/nginx/nginx.conf by default. The path may vary based on the environment that you use.

   http {
     # Set the variable $http_x_forwarded_for, which is used to record the value of X-Forwarded-For.
     log_format  main  '$remote_addr- $remote_user [$time_local] "$request" '
                         '$status $body_bytes_sent "$http_referer" '
                         '"$http_user_agent" "$http_x_forwarded_for"';
     
     # ...
   }
   

3. Run the sudo nginx -s reload command to reload the NGINX configuration file.

4. Obtain client IP addresses.

   You can check the NGINX log to determine whether the backend servers can obtain client IP addresses.

   Run the tail -n <the number of log entries to be returned> <log path> command. In the returned log, view the field corresponding to the $http_x_forwarded_for variable. The first IP address is the source IP address of the client.

   In this example, the default path of the NGINX log file (/var/log/nginx/access.log) is used.

   

   Note

   If you enable health checks for GA, we recommend that you return multiple log entries at a time or increase the health check interval to prevent the log entries that contain client IP addresses from being overwritten by health check log entries.

Method to obtain client IP addresses

You can use the client IP address preservation feature. After this feature is enabled, you need to select the method to obtain client IP addresses.

The supported method varies based on the type of the backend service.

• Automatic: Client IP addresses are directly passed to backend servers.

• Proxy Protocol: Proxy Protocol is required for backend servers to obtain client IP addresses.

  What is Proxy Protocol?

  Proxy Protocol is an Internet protocol which passes original connection information between proxy servers and backend servers.

  In most cases, a proxy server overwrites the request header that carries the source client IP address and then forwards requests to backend servers. The source client IP address and port are replaced by the IP address and port of the proxy server. In this scenario, the backend servers cannot obtain the original connection information.

  Proxy Protocol allows the proxy server to encapsulate the original connection information in a request header, which can be passed to backend servers. The backend servers can obtain the original connection information by parsing the request header encapsulated by Proxy Protocol. The original connection information includes the source IP address, source port, and transmission protocol.

  The original connection information obtained by backend servers can be further used in logging, access control, and traffic monitoring.

  Important

  • Proxy Protocol l takes effect only if it is enabled on both the proxy server and backend servers. If a backend server cannot parse Proxy Protocol headers but Proxy Protocol is enabled, parsing errors may arise on the backend server and compromise service availability.

  • Proxy Protocol can be used by GA listeners to pass client IP addresses to backend servers. After Proxy Protocol is enabled, a TCP header is added to the request without modifying the existing headers. The TCP header contains information such as the source IP address, destination IP address, source port, and destination port.

  • GA supports only Proxy Protocol v1. Proxy Protocol v1 supports only TCP. For more information, see The PROXY protocol.

Procedure

In this example, the Alibaba Cloud Linux 3 operating system and NGINX 1.20.1 are used. Adjust the configurations based on the environment that you use.

Method to obtain client IP addresses

You can use the client IP address preservation feature. After this feature is enabled, only the automatic retrieval method is supported by default. GA directly passes client IP addresses to backend servers.

To enable client IP address preservation for UDP listeners, take note of the following limits:

• Only pay-as-you-go GA instances allow you to enable client IP address preservation for UDP listeners.

  If your pay-as-you-go GA instance does not support client IP address preservation for UDP listeners, the instance may use an earlier version. Contact your account manager to upgrade your GA instance.

• The backend service type must be NLB, ECS, or ENI. By default, client IP address preservation is not supported by backend services of the NLB type. To use this feature, contact your account manager.

• You can obtain only IP addresses of IPv4 clients.

Procedure

In this example, the Alibaba Cloud Linux 3 operating system and NGINX 1.20.1 are used. Adjust the configurations based on the environment that you use.

1. Enable client IP address preservation.

   1. Log on to the GA console.

   2. On the Instances page, find the GA instance that you want to manage and click Configure Listener in the Actions column.

   3. On the Listeners tab, find the UDP listener that you want to manage and click Mod Lsnr in the Actions column.

   4. In the Configure Listener & Protocol step, click Next.

   5. In the Configure Endpoint Group step, select Preserve from the Preserve Client IP drop-down list, and then click Next.

      Retrieve Client IP Address is set to Automatically Retrieve Client IP by default.

      

   6. In the Configuration Review step, confirm the configurations and click Submit.

2. View the configuration file of the NGINX service.

   Run the nginx -t command to query the path of the configuration file. In most cases, the path of the configuration file is /etc/nginx/nginx.conf. The path may vary based on the environment that you use.

   Note

   If you want the backend service to process UDP traffic, make sure that the NGINX server supports the stream{} module.

   The stream{} module is added in NGINX 1.9.0 and later. However, this module is not installed by default. You must compile and install the stream{} module.

   #...
   
   # Process UDP traffic. Make sure that this module is added.
   stream {
       log_format main_stream '$remote_addr - [$time_local] '
                           '"$protocol" $status $bytes_sent $bytes_received '
                           '"$session_time"';
   
       access_log  /var/log/nginx/stream_access.log  main_stream;
   # Port 4000 is used in this example.
       server {
           listen 4000 udp;
           #...
       }
   }

3. Obtain client IP addresses.

   • Method 1: You can check the NGINX log to determine whether the backend servers can obtain client IP addresses.

     Run the tail -n <the number of log entries to be returned> <log path> command. In the returned log, view the field corresponding to the $remote_addr variable. The first IP address is the source IP address of the client.

     In this example, the log file path is /var/log/nginx/stream_access.log.

     

   • Method 2: You can capture data packets to determine whether the client IP addresses are obtained.

     Run the tcpdump udp port [listener port] -n -X -s 0 command to view client IP addresses in the captured packets.

     

Why do the backend servers fail to obtain client IP addresses after I enable client IP address preservation?

• If you use an HTTP or HTTPS listener and the backend server cannot record or process the X-Forwarded-For header, client IP address preservation does not take effect.

• If you use a TCP listener:

  • If the backend service is deployed on Alibaba Cloud but the backend servers are not deployed in a virtual private cloud (VPC) or the elastic IP address (EIP) is not associated with an instance of the VPC type, the backend servers cannot preserve client IP addresses.

  • If the backend service is deployed outside Alibaba Cloud, the backend servers must support the Proxy Protocol so that they can preserve client IP addresses.

    This is because GA uses Proxy Protocol to preserve client IP addresses after you enable client IP address preservation. Therefore, you must make sure that the backend servers support Proxy Protocol. Otherwise, the backend servers cannot parse the Proxy Protocol header.

  • If the client uses an IPv6 address to access an IPv4 backend service, the backend servers must support Proxy Protocol. Otherwise, the backend servers cannot preserve client IP addresses.

• If you use a UDP listener and the client uses an IPv6 address to access an IPv4 backend service, client IP address preservation does not take effect.

How do I disable client IP address preservation?

For HTTP listeners and HTTPS listeners, client IP address preservation is automatically enabled and cannot be disabled.

How do I obtain client IP addresses when IPv6 clients access IPv4 services of GA?

• HTTP and HTTPS listeners obtain the client IP addresses of IPv6 and IPv4 clients by using the X-Forwarded-For HTTP header.

• TCP listeners obtain the client IP addresses of IPv6 clients by using Proxy Protocol. This method requires that the backend servers support Proxy Protocol. Otherwise, access may fail.

• UDP listeners cannot obtain the client IP addresses of IPv6 clients.