CDN:Origin fetch troubleshooting

A CDN node, acting as a gateway, returns a 502 error (Bad Gateway) when it cannot obtain a valid response from the origin server. A failure at any of the following stages in the origin fetch path may return a 502 error:

• The origin server does not support HTTPS (it listens only on port 80), but CDN is configured to perform origin fetch over HTTPS.

• The origin port does not match the port on which the origin server actually listens.

• The origin server relies on SNI to select the certificate, but CDN does not carry an SNI value or carries an incorrect one.

• The SSL certificate of the origin server has expired or is invalid, or does not match the domain name.

• The origin server uses a self-signed certificate or a certificate issued by an internal CA, and TLS validation during origin fetch fails.

Troubleshooting steps:

• Check whether the origin server supports the current origin fetch protocol: Run curl -Iv https://origin-domain to verify the origin server directly. If the connection is refused or times out, the origin server does not support HTTPS. Change the origin protocol to HTTP. If the origin server supports HTTPS and the certificate is valid, you can choose HTTPS origin fetch or protocol Follow.

• Check whether the origin port matches the port on which the origin server listens: The default origin port is 443 for HTTPS and 80 for HTTP. If the origin server uses a custom port (1 to 65535), specify the corresponding port in the origin protocol configuration.

• Check the origin SNI configuration: When the origin server hosts multiple HTTPS sites on the same IP address, it relies on the SNI (Server Name Indication) field in the TLS handshake to select the corresponding SSL certificate. If CDN origin fetch does not carry an SNI value or the SNI value is incorrect, the origin server cannot match the correct certificate, the TLS handshake fails, and a 502 error is returned.

  Solution steps: Log on to the CDN console, choose Domain Management, select the target domain name, go to Origin Settings, and enable origin SNI. Enter the domain name that the origin server actually uses to provide services (usually the same as the Common Name of the origin certificate). Also set the origin host to the origin domain name.

  During origin fetch, CDN validates the SNI value against the Common Name in the certificate of the origin server. If the two values genuinely cannot match (for example, the origin server uses a certificate on a unified access layer), add the Common Name of the certificate to the Common Name whitelist.

  Note

  Origin SNI is used to select the certificate during the TLS handshake, while the origin host is used for virtual host routing at the HTTP layer. The two serve different purposes, but they are usually set to the same origin domain name.

• Check the validity of the SSL certificate on the origin server: Confirm that the certificate has not expired and has not been revoked, and that the domain names in the certificate include the origin domain name. Run curl -Iv https://origin-domain 2>&1 | grep -E "expire|subject|issuer" to view the validity period, issuer, and bound domain names of the certificate. If the certificate has expired or does not match the domain name, update the certificate on the origin server. If the origin server uses a self-signed certificate or a certificate issued by an internal CA, replace it with a certificate issued by a publicly trusted CA, or change the origin protocol to HTTP.

Solution: Change the origin protocol based on the actual configuration of the origin server: the origin server supports only HTTP → select HTTP (port 80 by default); the origin server supports HTTPS and the certificate is valid → select HTTPS (port 443 or a custom port); the origin server supports both protocols and the certificate is well maintained → you can select protocol Follow. For detailed steps, see Configure the origin protocol policy.

How do I troubleshoot a 502 error when the origin protocol is set to Follow?

When the origin protocol is set to Follow, CDN uses the same protocol as the client request for origin fetch: if a client uses HTTP, CDN uses HTTP for origin fetch; if a client uses HTTPS, CDN uses HTTPS for origin fetch. If a client accesses CDN over HTTPS but the origin server does not support HTTPS, the TLS handshake fails and the origin fetch request fails. This issue has the same cause as "How do I troubleshoot a 502 error returned during origin fetch?" and can be diagnosed with the same troubleshooting steps.

Solutions:

• Method 1: Change the origin protocol from Follow to HTTP so that CDN always uses HTTP for origin fetch.

• Method 2: Configure an SSL certificate on the origin server so that the origin server supports HTTPS access. For more information, see Configure the origin protocol policy.

A 504 error (Gateway Timeout) indicates that a CDN node cannot obtain a response from the origin server within the specified time during origin fetch.

Common causes include:

• The origin server responds slowly or the service is unavailable.

• Origin fetch requests are dropped by an intermediate network device or firewall (when SYN packets are dropped, it manifests as a connection timeout).

• The origin protocol or port is misconfigured, so the connection cannot be established (this usually manifests as a 502 error, but if the firewall silently drops packets instead of actively refusing them, it may also manifest as a 504 error).

• The origin read timeout is too short to cover the actual response time of the origin server.

Origin fetch timeouts fall into two phases, which point to different troubleshooting directions:

• Connection phase timeout: The timeout for a CDN node to establish a TCP connection with the origin server is 10 seconds. A timeout in this phase usually indicates that the origin server does not listen on the origin port, a firewall or security group drops the SYN packets from CDN origin fetch IP addresses, or severe packet loss exists on the network path.

• Read phase timeout: The connection has been established, but the origin server does not return a complete response within the origin read timeout (30 seconds by default). A timeout in this phase usually indicates that the origin server processes requests slowly, for example due to slow database queries, blocked backend applications, or high load on the origin server.

Troubleshooting steps:

1. Check whether the origin server is accessible: Run curl -I http(s)://origin-domain or access the origin server in a browser to check its response time and status code. If the origin server itself responds slowly or is inaccessible, fix the performance or availability issue on the origin server first.

2. Identify which phase the time is spent in: Run the following command:

   curl -o /dev/null -s -w "time_connect:%{time_connect} time_starttransfer:%{time_starttransfer} time_total:%{time_total}\n" http(s)://origin-domain/resource-path

   time_connect is the time taken to establish the TCP connection, time_starttransfer is the time taken to receive the first byte, and time_total is the total time.

   • If time_connect is already significantly large, troubleshoot the network and firewall as a connection phase issue.

   • If time_starttransfer is much greater than time_connect, the business processing on the origin server is slow. Optimize the origin server.

   • If time_total is much greater than time_starttransfer, the response body is too large or the egress bandwidth of the origin server is insufficient.

3. Check the origin protocol and port configurations: Make sure that the origin protocol and port configured in CDN match the configuration on which the origin server actually listens. If the origin port is incorrect or the origin server does not listen on that port, CDN returns a 504 error after the request times out.

4. Check the firewall or security group of the origin server: If the origin server throttles or blocks CDN origin fetch IP address ranges, some requests may time out. Policies that silently drop SYN packets manifest directly as connection phase timeouts in particular. Add the CDN origin fetch IP address ranges to the whitelist of the origin server (see What are the CDN origin fetch node IP addresses).

5. Check the quality of the network path between CDN nodes and the origin server: Network jitter, packet loss, or carrier routing issues may exist between CDN nodes and the origin server. Use tools such as MTR or traceroute to analyze the path (contact Alibaba Cloud technical support to initiate diagnostics from the CDN node side).

6. Increase the origin read timeout: If the response time of the origin server is close to the default of 30 seconds, you can increase the origin HTTP request timeout in the CDN console to reduce 504 errors. The value can be configured up to 150 seconds at most, but we recommend that it not exceed 60 seconds. Use this only as a temporary mitigation. The fundamental solution is still to optimize the response performance of the origin server.

A 503 error (Service Temporarily Unavailable) indicates that the origin server is temporarily unable to process requests. A 503 error received during CDN origin fetch is usually caused by the origin server side:

• The web service program on the origin server is abnormal, not started, or restarting.

• The load on the origin server is too high (CPU, memory, or connection counts are saturated).

• A per-IP request rate limit or concurrent connection limit is configured on the origin server.

• Security policies such as server security software (Yunsuo or SafeDog), a WAF, or a firewall on the origin server block CDN origin fetch IP addresses.

• The origin server has entered maintenance mode or is being deployed.

Troubleshooting steps:

1. Bind the hosts file to reproduce the issue by accessing the origin server directly: Modify the local hosts file to point the accelerated domain name to the IP address of the origin server, and then access the domain name. If direct access to the origin server also returns a 503 error, the issue is on the origin server side, and the CDN node itself can be ruled out.

2. Check whether the web service on the origin server is normal: Confirm that web service processes such as NGINX, Apache, or IIS are running and listening on the corresponding port (80/443 or a custom port). If a service process is abnormal or not started, restart the service and check the service logs to locate the cause.

3. Check the load and rate limit configuration of the origin server: High CPU, memory, or connection load on the origin server, or a per-IP request limit (such as the limit_req or limit_conn module of NGINX), may cause a 503 error to be returned for CDN origin fetch requests. Evaluate whether you need to scale out resources or adjust the rate limit thresholds based on your traffic volume.

   Note

   CDN origin fetch requests come in a concentrated manner from a limited set of origin fetch node IP addresses, and are easily mistaken for high-frequency access from a single IP when the origin server throttles by IP. We recommend that you set a higher rate limit threshold for CDN origin fetch IP address ranges, or exempt them directly.

4. Check whether security policies block CDN origin fetch IP addresses: Security policies such as the security group, firewall, WAF, or server security software on the origin server may identify CDN origin fetch IP addresses as abnormal traffic and block them. Look for records from CDN node IP addresses in the blocking logs, and add the CDN origin fetch IP address ranges to the whitelist (for how to obtain them, see What are the CDN origin fetch node IP addresses).

5. Refresh the CDN cache: If a 503 error is still returned after the origin server recovers, run a URL refresh. For status codes such as 500, 502, 503, and 504, the caching priority of CDN is: do not cache when the origin server returns Set-Cookie → cache according to the status code expiration time configured in the console if one is configured → otherwise cache according to the Pragma, Cache-Control, and Expires response headers of the origin server → cache for 1 second by default when none of the preceding conditions apply. Therefore, by default a 503 error does not cause persistent impact. However, if a long status code expiration time was configured for 5xx status codes in the console, the abnormal responses continue to be cached until they expire. In this case, refresh manually or set the cache time of 5xx status codes to 0 (see Configure expiration for HTTP status codes).

Symptom

Origin servers usually distinguish virtual sites by the Host request header. If the origin host configured in CDN does not match the domain name expected by the origin server, the origin server may return errors such as 404, 403, or 500.

Troubleshooting steps

1. Check whether the origin host matches the virtual host configuration of the origin server

   • NGINX: Check whether server_name contains the domain name configured as the origin host.

   • Apache: Check ServerName / ServerAlias in <VirtualHost>.

   • IIS: In IIS Manager, select the target website > Bindings, and check whether the "Host name" field matches the origin host.

2. Refresh the CDN cache

After you modify the origin host configuration, run a URL refresh to clear any error responses that may have been cached.

Symptom: The website returns a "Too many redirects" or "ERR_TOO_MANY_REDIRECTS" error, or resources such as images, CSS files, and JavaScript files fail to load.

Cause: The origin server actively responds with a forced HTTP-to-HTTPS redirect (a common rule in services such as aaPanel, WAF, and NGINX), but the CDN origin protocol is set to HTTP. As a result, the request path forms a loop:

Client → CDN → CDN performs origin fetch over HTTP (port 80) → the origin server returns a 301 redirect to HTTPS → if origin 301/302 redirect follow is not configured, CDN returns the 301 to the client → the browser follows the redirect to HTTPS → origin fetch through CDN over HTTP happens again → the origin server returns another 301 → ... the browser redirects repeatedly and eventually reports ERR_TOO_MANY_REDIRECTS. If origin 301/302 redirect follow is enabled, CDN nodes follow the redirects repeatedly on the origin fetch path and return the 301 to the user after the follow count limit is reached, which also forms a loop. For more information about the redirect follow feature, see Configure 301/302 redirection.

Solutions (choose one):

• Solution A: Make CDN use HTTPS for origin fetch. Prerequisite: the origin server has a valid SSL certificate and listens normally on port 443. Change the origin port to 443 and set the origin protocol to HTTPS. If the origin server listens for multiple domain names, also set the origin SNI and origin host to the accelerated domain name or the origin domain name.

• Solution B: Disable the forcible redirect on the origin server and keep HTTP communication. Log on to the origin server and disable the rule that forcibly redirects HTTP to HTTPS. Keep the CDN origin protocol as HTTP and the origin port as 80. The connection between clients and CDN can still use HTTPS, as long as CDN and the origin server agree on the same protocol.

Additional operation (recommended regardless of which solution you choose): Run a URL refresh task to clear cached redirect responses so that the new configuration takes effect immediately. After the configuration is modified, it needs to be distributed to all nodes across the network, which usually takes a few minutes. During this period, some nodes may still return the old redirect response. For the default cache policies of CDN for various status codes, see Configure expiration for HTTP status codes.

This entry applies only to scenarios in which the redirect loop appears only after you configure the origin host or origin SNI. If the redirect loop occurred before you configured them, see the preceding entry "What do I do if a redirect loop (ERR_TOO_MANY_REDIRECTS) occurs during CDN origin fetch after the origin server is configured to redirect HTTP to HTTPS?" The root cause is still that the origin server has enabled a forced HTTP-to-HTTPS redirect while CDN performs origin fetch over HTTP. After you modify the origin host or origin SNI, the origin server matches the expected site and the forced redirect rule starts to take effect, so the loop appears. Troubleshoot as follows:

• Disable the forcible HTTPS redirect on the origin server: Log on to the management console of the origin server (for example, aaPanel), go to the HTTPS settings, and disable the "Force HTTPS" or "HTTP to HTTPS redirect" option.

• Make sure that the origin host and origin SNI are consistent and correct: On the origin settings page, set the origin host and origin SNI to the correct domain name (usually the origin domain name) and keep the two values consistent so that origin fetch requests meet the expectations of the origin server.

• Refresh the CDN cache: After you adjust the configuration, refresh the CDN cache to clear cached redirect responses.

This error indicates that the access permission of the OSS bucket is private, and requests without a signature cannot read objects in the bucket. A private bucket provides access authentication and prevents unauthorized requests from consuming traffic. Therefore, we do not recommend changing the bucket to public read just to get rid of this error.

Solution: Enable the Configure origin fetch from a private OSS bucket feature for the accelerated domain name. After the feature is enabled, CDN automatically uses the service role AliyunCDNAccessingPrivateOSSRole to carry a signature when accessing the private bucket, and end users do not need additional signatures when accessing resources through CDN. Operation path: CDN console > Domain Management > target domain name > Origin Settings > Origin fetch from private OSS buckets.

If your OSS bucket is encrypted with Key Management Service (KMS), you must grant the CDN origin fetch role additional permissions to use the KMS key. Otherwise, CDN cannot decrypt and access these files, and the error This request is forbidden by kms. is returned.

Solution:

1. Log on to the RAM console. In the left-side navigation pane, choose Identities > Roles.

2. In the role list, find the AliyunCDNAccessingPrivateOSSRole role and click Grant Permission.

   Note

   If you cannot find the role, the origin fetch from private OSS buckets feature has never been enabled. First enable the feature as described in Configure origin fetch from a private OSS bucket. The role is created automatically. Then return to perform this step.

3. In the permission policy list, select System Policy, search for and add AliyunKMSCryptoUserAccess, and then click Grant permissions.

4. Use the Refresh and Prefetch feature. After the refresh task is complete, access the resource again.

This issue occurs when all of the following three conditions are met: the OSS bucket permission is private, OSS static website hosting is enabled, and Configure origin fetch from a private OSS bucket is enabled in CDN. When you access the root path of the accelerated domain name (for example, https://example.com/), a 403 Forbidden error is returned, and the response header contains x-tengine-error: You are forbidden to list buckets.

Cause: The origin fetch from private OSS buckets feature of CDN conflicts with the default homepage configuration of OSS static website hosting.

Solutions:

• Solution 1: If you do not need OSS static website hosting, disable the static website hosting configuration for the bucket. For instructions, see Static website hosting.

• Solution 2: If you need to keep static website hosting, configure a URI rewrite rule in CDN to prevent origin fetch requests that target the root directory: set Path to Be Rewritten to ^/$, set Target Path to /index.html, and set Flag to Redirect. After the rule is configured, when a client requests the root path, the CDN node returns a 302 redirect instructing the client to request /index.html. For detailed steps, see Rewrite access URLs.

Symptom: The origin server uses 302 redirects to serve the corresponding interface based on the client device type. After CDN is enabled, the 302 response is cached when the first user accesses the resource. Users on other device types that access the same URL then hit the cached 302 page of the first user, so the device adaptation feature stops working.

Solution A (recommended): Do not cache the 302 response. Configure CDN not to cache the first requested URL, and cache the page after the 302 redirect instead. You can configure the origin server not to cache the initial page (the no-cache policy of the origin server has a high priority for CDN). As long as the response of that page contains any of the following response headers, the page is not cached:

• Cache-control:no-cache, no-store

• Cache-control:max-age=0

• pragma:no-cache

• Cache-control:private

Solution B: Configure CDN not to cache the initial URL. If you cannot modify the response headers of the origin server, combine the CDN cache configurations for directories and file name extensions with their priorities to set the cache time of the initial redirect URL to 0, while other URLs continue to be cached normally.

Solution C: Use a custom cache key to distinguish device types. If you want to keep the caching capability instead of fetching from the origin every time, configure a custom cache key that includes the device type dimension, so that PC and mobile devices each have an independent cache copy. For the configuration method, see Custom cache key.

A possible cause is that the origin server relies on URL parameters or a specific Host header for its logic, while the default behavior of CDN may cause parameters to be lost or the Host header to mismatch. Troubleshoot as follows:

1. Check the URL parameter configuration in the cache rules: If the origin server relies on URL parameters for redirects or logic, pay attention to the "Ignore Parameters" feature of CDN. Ignore Parameters only affects the cache key. Origin fetch requests still carry the full URL parameters. Therefore, the issue is not that "the origin server cannot process the request because parameters are lost," but that "requests with different parameters hit the same cache and receive content that does not belong to the current parameters." We recommend that you disable Ignore Parameters, or retain the parameters that affect business logic, so that requests with different parameters are cached independently (see Ignore parameters).

2. Set the origin host to the domain name expected by the origin server: This ensures that the origin server correctly identifies the request host header and processes the redirect logic.

3. Run a directory refresh or URL refresh task: After you modify the configuration, run a directory refresh or URL refresh task to apply the new configuration.

Symptom: Large file downloads are interrupted, resumable downloads fail, or video seeking returns an error or plays from the beginning.

Cause: After Range origin fetch is enabled, CDN nodes send slice requests with a Range header to the origin server. If the origin server does not support Range requests (it ignores the Range header and returns 200 with the full content), or the returned Content-Range does not match the requested range, cache exceptions or client request failures may occur.

Troubleshooting steps:

1. Verify whether the origin server supports Range requests: Run curl -I -H "Range: bytes=0-1023" http(s)://origin-domain/resource-path. If 206 Partial Content is returned with a correct Content-Range header, the origin server supports Range. If 200 OK is returned with the complete file, the origin server ignores Range requests.

2. Adjust the Range origin fetch configuration based on the capability of the origin server: If the origin server does not support Range, first modify the origin server so that it responds with 206 slices correctly, or disable the Range origin fetch feature of CDN first, to avoid CDN sending slice requests that the origin server cannot handle. Configuration path: CDN console > Domain Management > target domain name > Video Settings > Range Origin Fetch. This switch is disabled by default. For more information, see Configure range origin fetch.

3. Check whether the response headers of the origin server are stable: Range origin fetch requires the origin server to return a stable Content-Length, ETag, and Last-Modified for the same resource, so that CDN can determine that the slices belong to the same version of the file. If the origin server generates content dynamically and cannot provide these response headers, Range origin fetch is unreliable.

4. Check whether cached slices have been cleared: When Range origin fetch is used, if a CDN node receives a non-206 status code from the origin server, it deletes the cached slice files (origin fetch timeouts do not cause deletion). Therefore, if the origin server intermittently returns 5xx responses, the cached slices are repeatedly cleared. This manifests as downloads being interrupted repeatedly and abnormally high origin fetch traffic. For more information, see Configure expiration for HTTP status codes.

Symptom

When accessed through CDN, the page is garbled, or the browser reports a decoding failure (ERR_CONTENT_DECODING_FAILED).

Cause

• The origin server returned compressed content (gzip/br) without setting the Content-Encoding response header. CDN compresses the already-compressed content again before returning it, causing a decoding exception on the client.

• The Content-Encoding declared by the origin server does not match the actual encoding.

Troubleshooting steps

1. Compare the response headers of the origin server and CDN

# Direct access to the origin server
curl -I -H "Accept-Encoding: gzip" https://<origin-domain>/<resource-path>

# Access through CDN
curl -I -H "Accept-Encoding: gzip" https://<accelerated-domain>/<resource-path>

Check whether Content-Encoding, Content-Length, and Content-Type are consistent on both sides.

2. Check the intelligent compression configuration of CDN

If the origin server already returns compressed content (the response headers contain Content-Encoding: gzip), CDN should not compress it again. If double compression occurs, check whether "Intelligent Compression" is enabled in the CDN console, or disable compression on the CDN side and let the origin server handle compression entirely.

3. Fix the response headers of the origin server

Make sure that whenever the origin server returns compressed content, it also sets the correct Content-Encoding header. Otherwise, CDN cannot identify that the content is already compressed.

Symptom

When different users access the same page, one user receives session information of another user (such as a mixed-up login state), or the login state expires immediately after login.

Cause

The origin server returns a Set-Cookie header in the response of a dynamic page. If this response is cached by CDN, other users who hit the cache receive a Cookie that does not belong to them, which mixes up session information.

Solution

1. Configure the origin server not to cache responses that contain Set-Cookie

Add Cache-Control: no-store or Cache-Control: private to dynamic pages on the origin server to prevent CDN from caching responses that contain user-specific information.

2. Configure cache rules on the CDN side

In the CDN console, configure a "Do Not Cache" rule for dynamic pages (such as .php, .jsp, or the /api/ path) to make sure that these requests always go back to the origin server.

3. Use the Modify Inbound Response Headers feature of CDN to remove the response header

If you confirm that Set-Cookie is meaningless for CDN caching scenarios (such as tracking cookies), you can configure the Modify Inbound Response Headers feature on the CDN side to remove the Set-Cookie response header before caching. Make sure that this does not affect business logic.