Elastic Compute Service:Use PrivateLink to reduce unnecessary public network communication

Use PrivateLink for cross-VPC access within the same zone

Description: A network service, such as a web service, is deployed on an ECS instance in VPC2. A PrivateLink connection is created between VPC1 and VPC2. This allows the network service to be accessed from VPC1 using a private IP address (endpoint).

Note: To use PrivateLink, the endpoint (service consumer) and the endpoint service (service provider) must be in the same zone. More specifically, the zones where the endpoint is deployed must be a subset of the zones where the endpoint service resources are deployed. For example, if an endpoint service is deployed in Zones A and B, the endpoint can be created only in Zone A or Zone B.

Core steps:

1. Server-side (service provider) configuration

   1. Create a load balancer: In VPC2, create an SLB instance that supports PrivateLink, such as a private Classic Load Balancer (CLB) or Network Load Balancer (NLB). Configure a backend server group and a listener.

   2. Create an endpoint service: In the PrivateLink console, create an endpoint service. Associate it with the SLB instance created in the previous step.

   3. Configure a service whitelist (only for cross-account access): If VPC1 belongs to a different Alibaba Cloud account, add the account's UID to the service whitelist of the endpoint service.

   4. Accept the connection request: After the client initiates a connection, navigate to the Connections tab of the endpoint service. Find the pending connection request and accept it. You can also set the service to automatically accept connections during creation.

2. Client-side (service consumer) configuration

   1. Create an endpoint: In VPC1, create an endpoint. During creation, select the endpoint service created on the server side.

   2. Configure the network: Select a vSwitch and a security group in VPC1 for the endpoint. This controls which resources in VPC1 can access the endpoint.

   3. Obtain the endpoint details: After the endpoint is created, a private domain name and an IP address are generated.

   4. Initiate access: After the service provider accepts the connection, applications in VPC1 can use this domain name or IP address to access the service in VPC2.

For more information, see Access Alibaba Cloud services over a private network.

Use Proxy Protocol to control accessible VPCs and their private IP addresses

When you provide services through PrivateLink, the server can identify the true source of network requests, such as which endpoint or VPC they come from. This allows the service to set security rules. For example, you can restrict a user account to be used only within a legitimate private network, which means it accepts access only from a legitimate VPC. Even if the account credentials are leaked, this prevents the account from being used to access the service from the Internet or an unauthorized network.

Follow the steps in Use PrivateLink for cross-VPC access within the same zone, and then enable the server-side application to identify the source VPC of a request and implement access control based on this information.

1. Enable Proxy Protocol v2: Enable Proxy Protocol v2 on the server-side NLB listener and subscribe to the VPC ID and endpoint ID.

   # Use the Alibaba Cloud CLI to enable Proxy Protocol v2 for a specified listener and subscribe to the VPC ID and endpoint ID
   aliyun nlb UpdateListenerAttribute \
        --ListenerId lsn-xxxxxxxxxxxxxxxx \
        --ProxyProtocolEnabled true \
        --ProxyProtocolV2Config '{"Ppv2VpcIdEnabled":true,"Ppv2PrivateLinkEpIdEnabled":true}' \
        --RegionId cn-hangzhou

   --ListenerId: The ID of the listener created for the web service in the example.

   --ProxyProtocolEnabled: true indicates that the Proxy Protocol is enabled. Note: This protocol passes client connection information in the first packet after the TCP handshake. The backend service must support this protocol (see Step 2 for configuration). Otherwise, the service will be interrupted.

   --ProxyProtocolV2Config: Adds Alibaba Cloud-defined TLV fields. In this example, it subscribes to the source network's VPC ID and the PrivateLink endpoint ID.The corresponding TLV parsing is as follows:

   	Meaning/Content	Original Content
   Protocol header	Fixed signature of the Proxy Protocol	0d0a0d0a000d0a515549540a
   	Version 2, PROXY command	21
   	TCP/IPv4	11
   	The following data is 84 bytes long	0054
   Address information	Source IP: 10.0.0.14	0a00000e
   	Destination IP: 10.0.0.15	0a00000f
   	Source port: 59074	e6c2
   	Destination port: 80	0050
   TLV1	Type 03	03
   	4 bytes	0004
   	0764b56b	0764b56b
   TLV2	Type e1 (custom type)	e1
   	24 bytes	0018
   	PrivateLink endpoint ID: ep-bp1i288487e586152d4b	0265702d6270316932383834383765353836313532643462
   TLV3	Type e1 (custom type)	e1
   	26 bytes	001a
   	VPC ID: vpc-bp179qeke0wzo1mr8bxhl	017670632d627031373971656b6530777a6f316d72386278686c
   TLV4	04 (PP2_TYPE_NETNS), used to pass the network namespace	04
   	6 bytes	0006
   	000000000000	000000000000

2. If your web service uses NGINX, configure NGINX to support the Proxy Protocol as follows:

   (NGINX Plus R16 or later, or open source NGINX 1.13.11 or later, supports Proxy Protocol v2)

   server {
       listen 80 proxy_protocol;   # Enable Proxy Protocol v1/v2
       # Other configurations...
   }

   With this method, nginx automatically parses the Proxy Protocol TLV on port 80, but the standard version of nginx only supports fetching the IP address from the TLV.

   A Lua extension is required to process custom VPC IDs and PrivateLink endpoint IDs from Alibaba.

   The following code is an example of a Lua plugin for nginx.conf that handles TCP connections by reading packets from a socket and parsing the TLV buffer.

   Store the PrivateLink endpoint ID in the request context. For /login requests, check that this ID is correct. The following is a Lua code example:

   stream {
     server{
     listen 80;  # Do not enable proxy_protocol. Let the Lua code below handle it.
     preread_by_lua_block {
        -- 1. Try to read the fixed 16 byte protocol header
        local sock = ngx.req.socket(true)
        local header, err = sock:receive(16)
   
        if not header then
           ngx.log(ngx.ERR, "failed to receive proxy protocol header: ", err)
           return
        end
   
        -- 2. Check the protocol signature (12 bytes)
        if string.sub(header, 1, 12) ~= "\x0D\x0A\x0D\x0A\x00\x0D\x0A\x51\x55\x49\x54\x0A" then
            -- If it is not the Proxy Protocol, put the read data back into the buffer to avoid corrupting the original request
             sock:setreused(header)
                   return
       end
   
       -- 3. Parse the rest of the protocol header
       local ver_cmd = string.byte(header, 13) -- Version and command
       local family = string.byte(header, 14) -- Address family and protocol
       local len = string.byte(header, 15) * 256 + string.byte(header, 16) -- Total length of the variable part
       if len == 0 then
          -- No address or TLV information, return directly
          return
       end
   
       -- 4. Read the variable-length part (contains address and TLV)
       local variable_part, err = sock:receive(len)
       if not variable_part or #variable_part < len then
          ngx.log(ngx.ERR, "failed to read variable part (address + tlvs): ", err)
          return
       end
   
       -- 5. Determine the length of the address information based on the address family
       local addr_len = 0
       local protocol = family & 0x0F
       local address_family = family >> 4
   
       if address_family == 1 then -- AF_INET (IPv4)
           addr_len = 12
       elseif address_family == 2 then -- AF_INET6 (IPv6)
           addr_len = 36
       elseif address_family == 3 then -- AF_UNIX
           addr_len = 216
       end
   
       -- 6. Separate the address information and TLV data
       local address_block = string.sub(variable_part, 1, addr_len) -- You can parse the address information if needed
       local tlv_string = string.sub(variable_part, addr_len + 1)
   
       -- 7. Parse the TLV data
       local tlvs = {}
       local pos = 1
       while pos <= #tlv_string do
           -- Make sure there are at least 3 bytes (Type, Length)
           if pos + 2 > #tlv_string then
               ngx.log(ngx.ERR, "malformed TLV data: not enough bytes for type and length")
               break
           end
   
           local tlv_type = string.byte(tlv_string, pos)
           local tlv_length = string.byte(tlv_string, pos + 1) * 256 + string.byte(tlv_string, pos + 2)
       
           -- Make sure the length of the value does not exceed the boundary
           if pos + 2 + tlv_length > #tlv_string then
               ngx.log(ngx.ERR, "malformed TLV data: length exceeds available data")
               break
           end
       
           local tlv_value = string.sub(tlv_string, pos + 3, pos + 2 + tlv_length)
                       
           table.insert(tlvs, {
               type = tlv_type,
               length = tlv_length,
               -- Encode the binary value in Base64 for easy storage and viewing
               value = ngx.encode_base64(tlv_value) 
            })
            pos = pos + 3 + tlv_length
      end
   
      -- 8. Cache the parsed TLV in ngx.ctx
      -- ngx.ctx is unique for each request and is the standard way to pass data
     if #tlvs > 0 then
         ngx.ctx.proxy_protocol_tlvs = tlvs
         -- Example: If you want to log it, you can use the $proxy_protocol_tlvs variable in log_format in the http section
         -- You need to define `lua_set $proxy_protocol_tlvs 'return ngx.var.proxy_protocol_tlvs_json';` in the http block
         -- and `ngx.var.proxy_protocol_tlvs_json = cjson.encode(ngx.ctx.proxy_protocol_tlvs)` in `preread_by_lua_block`
     end
     }
     proxy_pass localhost:8080;   # Pass subsequent HTTP requests to the HTTP server
     }
   }
   
   http {
       server {
           listen 8080;
   
           # Dedicated location to handle /login requests
           location /login {
               access_by_lua_block {
                   local expected_value = "some value"
                   -- Get the value of the second TLV (Base64-encoded) passed from the stream layer from the NGINX variable
                   local privateLinkEndId = ngx.var.pp_tlv2_value
                   
                   -- Deny the request if the value does not meet the expectation
                   if received_value ~= expected_value then
                       ngx.log(ngx.ERR, "Access to /login denied: TLV value mismatch. Expected '", 
                               expected_value, "', got '", received_value, "'")
                       return ngx.exit(ngx.HTTP_FORBIDDEN)
                   end
                   -- If the verification is successful, the request continues to be processed
               }
               
               # After successful verification, forward the request to the backend Python program
               proxy_pass http://your_python_backend;
           }
   
           # Handle all other requests
           location / {
               # Here, you can decide whether to allow access based on your needs
               # For example, forward directly to the backend
               proxy_pass http://your_python_backend;
           }
       }
   
       # Your Python backend service upstream definition
       upstream your_python_backend {
           server 127.0.0.1:5000; # Assume your backend program runs on port 5000
       }
   }