This topic provides answers to some frequently asked questions about how to add web services to Web Application Firewall (WAF) 3.0.
Overview
Issues that may occur when web services are added to WAF
Issues that may occur on origin servers after web services are added to WAF
I cannot find the CLB, NLB, or ECS instance to add on the Website Configuration page. What do I do?
Problem description
You cannot find the Classic Load Balancer (CLB), Network Load Balancer (NLB), or Elastic Compute Service (ECS) instance that you want to add to WAF on the Website Configuration page.
Solution
Cause | Operation |
The CLB, NLB, or ECS instance does not meet the requirements. | Check whether the CLB, NLB, or ECS instance meets the requirements specified in the "Limits" section in the following topics: Enable WAF protection for a Layer 7 CLB instance, Enable WAF protection for a Layer 4 CLB instance, Enable WAF protection for an NLB instance, and Enable WAF protection for an ECS instance. |
The required listener is not added to the CLB instance. |
|
The CLB, NLB, or ECS instance is not synchronized to WAF. |
|
When I add an HTTPS listener port of a CLB instance to WAF, the system prompts that the certificate required for instance is incomplete. What do I do?
Problem description
When you add an HTTPS listener port of a CLB instance to WAF, WAF checks the source of the certificate configured for the port. The following error message appears: The CLB certificate whose port number is {port} is incomplete. Go to the SLB console and select a certificate that is from Certificate Management Service.
Cause
The certificate is not purchased by using Alibaba Cloud Certificate Management Service and is not uploaded to Certificate Management Service.
The certificate configured for the HTTPS listener port of the CLB instance is uploaded in the CLB instance. In this case, the certificate cannot be automatically synchronized to Certificate Management Service. However, WAF obtains certificate information only from Certificate Management Service. As a result, WAF cannot obtain the complete information of the certificate and the error message appears.
The certificate was uploaded to Certificate Management Service but was manually deleted. In this case, WAF cannot obtain the information about the certificate from Certificate Management Service.
Solution
Upload your certificate to Certificate Management Service. For more information, see Upload an SSL certificate.
Add the certificate in the CLB console and select Alibaba Cloud Certificates for Select Certificate Source. For more information, see Use a certificate from Alibaba Cloud SSL Certificates Service.
In the CLB console, select the added certificate for your port. For more information, see Step 2: Configure an SSL certificate.
WAF returns HTTP 502 status codes after web services are added to WAF. What do I do?
Problem description
When you access the web services that are added to WAF, WAF returns HTTP 502 status codes. Logs are queried, and the results include requests for which WAF returns HTTP 502 status codes.
Cause and solution
Scenario 1: HTTP 5XX status codes are occasionally returned after you enable WAF protection for a Layer 7 CLB instance
Current network architecture
Cause analysis
The timeout period of idle connections from WAF to CLB is 3,600 seconds, equivalent to 1 hour.
If no data is transmitted over a connection from WAF to CLB within 1 hour, WAF automatically closes the connection.
The timeout period of idle connections from CLB to WAF is 15 seconds.
In this case, WAF serves as the clients of CLB. If no data is transmitted over a connection from CLB to WAF within 15 seconds, CLB automatically closes the connection.
In extreme cases, WAF returns a back-to-origin request to CLB over a persistent connection at the point in time when the connection is aged. CLB ages a persistent connection if no data is transmitted over the connection within 15 seconds. In this case, CLB does not store information about the persistent connection and sends an RST packet to terminate the connection. In this case, WAF logs the request by using an HTTP 502 status.
Solution
Change the value of the Timeout Period of Idle Keep-alive Requests parameter in the configuration of the CLB instance to a value that is shorter than the timeout period of idle connections from CLB to WAF. For example, you can change the value to 14 seconds.
Scenario 2: HTTP 502 status codes are returned if URIs are excessively long
Current network architecture
Cause analysis
Layer 7 CLB serves as the next hop when WAF forwards traffic. However, CLB supports only requests whose Uniform Resource Identifiers (URIs) are up to 32 KB in length. If the URI length of requests exceeds the upper limit, CLB rejects the requests. In this case, CLB records HTTP 414 status codes in logs, and WAF returns HTTP 502 status codes.
Solution
Reduce the URI length. For large amounts of data, use the POST method to transmit data.
Scenario 3: HTTP 502 status codes are occasionally returned by multiple Layer 4 CLB instances to which WAF forwards back-to-origin requests
Current network architecture
In the current network architecture, web services are added to WAF in reverse proxy mode. WAF forwards back-to-origin requests to multiple Layer 4 CLB instances. The ECS instances serve as backend servers for the CLB instances and listen to the same port.
Cause analysis
If an ECS instance serves as a backend server for multiple Layer 4 CLB instances and the CLB instances use the same backend server port, the following issue may occur: If multiple clients concurrently request the web service provided by CLB and the requests are forwarded by the same node of WAF to CLB, connections to the web service may fail or time out because the same back-to-origin IP address is used and causes conflicts at the TCP layer.
Case 1: Five-tuple conflicts and TCP stream disorder
When WAF protects multiple CLB instances, the instances may receive requests from the same back-to-origin IP address of WAF.
When WAF forwards a back-to-origin request to CLB Instance 1, the connection is expressed as WIP:CPORT->VIP1:VPORT1. After the request reaches the required ECS instance, the connection becomes WIP:CPORT->DIP:DPORT.
When WAF forwards a back-to-origin request to CLB Instance 2, the connection is expressed as WIP:CPORT->VIP2:VPORT2. After the request reaches the required ECS instance, the connection also becomes WIP:CPORT->DIP:DPORT.
The connection fails to be established because of conflicts in the sequence numbers and status of the two TCP connections on the backend server. The backend server considers the two TCP connections to have the same five-tuple: TCP:WIP:CPORT:DIP:DPORT. This type of five-tuple conflict may cause SYN packets to be discarded.
Case 2: Response disorder
In a complete request path, the CLB instance that initiates the request differs from the CLB instance that returns the response.
When WAF sends a SYN packet to CLB Instance 2, the connection is expressed as WIP:CPORT->VIP1:VPORT1. After the request reaches ECS Instance 2, the connection becomes WIP:CPORT->DIP:DPORT.
ECS Instance 2 verifies that the SYN packet is valid and returns a SYN-ACK packet. In this case, ECS Instance 2 maintains an existing connection to CLB Instance 1 in the TIME-WAIT state. The connection is expressed as TCP:WIP:CPORT:DIP:DPORT.
ECS Instance 2 reuses the existing connection and returns the SYN-ACK packet to CLB Instance 1. However, CLB Instance 1 cannot find a session with WIP:CPORT->VIP1:VPORT1 and returns an RST packet to WAF and ECS Instance 2. As a result, WAF returns an HTTP 502 status code.
Solutions
(Recommended) Solution 1
Adjust the network architecture. For example, replace Layer 4 CLB instances with Layer 7 CLB instances to prevent request forwarding from different Layer 4 CLB instances to the same backend server on the same port.
Solution 2
Replace CLB instances with NLB instances. When you configure the NLB instances, turn off Client IP Preservation to resolve five-tuple conflicts. Then, enable the Proxy protocol for the backend servers to obtain the originating IP addresses of clients. For more information, see Enable the Proxy protocol.
Procedure
Log on to the NLB console. In the top navigation bar, select the region in which your NLB instance resides.
In the left navigation pane, choose .
Find your server group and click Modify Basic Information in the Actions column. In the dialog box that appears, turn off Client IP Preservation and click Save.
(Not recommended) Solution 3
Submit a ticket to enable the FULLNAT mode. In this mode, when multiple CLB instances forward requests to the same backend server on the same port, the source addresses are modified to ensure unique five-tuples for each connection and prevent five-tuple conflicts. We recommend that you enable the FULLNAT mode for a CLB listener.