Problem
After your website is accelerated by Alibaba Cloud CDN, its content delivery may be slow. Many factors can cause slow content delivery. This topic describes how to troubleshoot the error.
Solution
Note:
Before you perform high-risk operations, such as modifying the configurations or data of Alibaba Cloud instances, we recommend that you check the disaster recovery and fault tolerance capabilities of the instances to ensure data security.
Before you modify the configurations or data of an instance, such as an Elastic Compute Service (ECS) instance or an RDS instance, we recommend that you create snapshots or enable backup for the instance. For example, you can enable the log backup feature for an RDS instance.
If you granted access permissions on sensitive information or submitted sensitive information, such as usernames and passwords, in the Alibaba Cloud Management Console, we recommend that you modify the information at the earliest opportunity.
If content delivery is slow after your website is accelerated by Alibaba Cloud CDN, you can refer to the following content to resolve the issue.
Troubleshooting
Before you troubleshoot the issue, you need to understand how Alibaba Cloud CDN works. For more information, see What is Alibaba Cloud CDN? This information helps you identify and analyze the possible causes of the issue. Alibaba Cloud CDN adds points of presence (POPs) to the existing network and caches resources retrieved from the origin server to the POPs closest to users. When the resources are requested again, Alibaba Cloud CDN returns the resources directly from the cache instead of retrieving them from the origin server. This helps reduce origin requests and improve page loading. The following items describe the possible causes behind the low access speed:
Network issues occur on the client, such as insufficient downstream bandwidth and DNS configuration errors.
Network connection between the client and the POP is poor and access latency is high.
An error occurs on the POP, causing a low response speed.
The size of the resource is large and the resource requires a longer period of time to be downloaded.
Network connectivity between POPs and the origin server is poor.
The response speed of the origin server is low.
Take note of symptoms to inform subsequent troubleshooting.
Determine if content delivery is slow for all users, individual users, users in a specific region, or users of a specific Internet service provider (ISP). Alibaba Cloud provides services such as Application Real-Time Monitoring Service (ARMS) and CloudMonitor. You can use the services for more accurate troubleshooting that targets a specific region or ISP:
If content delivery is slow for only a few users, the network of these users may be abnormal.
Determine whether users with problems are centralized. For example, content delivery may be slow for a large number of China Mobile users in a specific city, but normal for China Unicom or China Telecom users. In this case, the network of the ISP in the city may be abnormal. You can use detection tools for the city to check the network conditions.
If content delivery is slow for all users, the response or configuration of the origin server is likely the issue. It is very unlikely that all POPs or networks in all regions have problems at the same time. For example, you can check whether the acceleration region is incorrectly selected or whether the origin server responds slowly to dynamic requests or requests for resources that cannot be cached.
Check whether resources with slow or abnormal responses are cached on POPs.
If a request hits the cache, the cached resource is returned to the client and is not retrieved from the origin server. In this case, slow content delivery is not related to the origin server.
If a request does not hit the cache, you need to check whether slow content delivery is caused by the route between the client and the POP or caused by the origin server.
Observe indicators
When you use Alibaba Cloud CDN to accelerate content delivery, you can obtain scenario-specific indicators in addition to general indicators. You can learn how Alibaba Cloud CDN accelerates content delivery and how it is applied to your services by referencing these indicators. This helps you make adjustments and business decisions. For more information about Alibaba Cloud CDN indicators, see Performance indicators.
Collect information
A complete HTTP or HTTPS request process involves DNS resolution, TCP connection, SSL handshake (for HTTP requests), client request sending, and server responding. Understanding the process helps you better analyze the issue. Therefore, it is necessary to collect the following information on the clients:
Collect network conditions of clients and IP addresses of POPs
Run the ping command to connect to the accelerated domain name. Check whether the domain name is correctly resolved to Alibaba Cloud CDN, whether the network between the client and the POP is accessible, and whether the network latency is high. If you cannot connect to the accelerated domain name, you need to diagnose the network path. For more information, see Test network paths when packet loss or connection failures occur after the ping command is run.
Collect IP addresses and local DNS servers of clients
The POP routing policy is based on the local DNS server of the client. Therefore, you must confirm that the local DNS server is correctly configured. You can visit https://cdn.dns-detect.alicdn.com/ to obtain the IP address and DNS server of the client.
Identify URLs that have slow access speeds
Turn on the developer mode in the browser, click the Network tab, enter a URL, and then view all HTTP requests sent by the browser on the Network tab. Click the Time column to sort results by time, and view resources that have slow response speed. In the Domain column, find the URLs of these resources.
In most cases, if a website loads a large number of resources, some resources may not be accelerated by Alibaba Cloud CDN. Resources that are not accelerated by Alibaba Cloud CDN have slow access speeds, and resources that are accelerated by Alibaba Cloud CDN have fast access speeds. However, the loading of resources that are not accelerated can slow down the response speed of the website as a whole. Therefore, you can sort the URLs by time to determine which URLs have slow access speeds.
Collect HTTP requests headers and response headers
On the Network tab, find a resource that has a low access speed and click the value in the Name column. On the Headers tab, you can view the General, Response Headers, and Request Headers information of the request for the resource. The request headers and response headers allow you to determine whether a request is a static request and whether it hits the cache.
If you access a resource over a 4G network on a mobile phone and the response is slow, you need to capture packets on the mobile phone to obtain information. This operation may be difficult for regular users. Instead, you can turn on the personal hotspot of the mobile phone, connect to the hotspot from a PC, and collect information on the PC.
Collect the Timing information of HTTP requests
On the Timing tab, view information about the time spent on each part of the resource request process.
Common cases
After you understand how Alibaba Cloud CDN works and the HTTP request process, you can perform a preliminary analysis based on the symptoms. Then, you can use the information that is collected on clients to troubleshoot or identify the issue. The following section describes some common cases.
Case 1: poor network quality between the client and the POP
A client uses the ping command to connect to the accelerated domain name. The ping command screen shows that the network latency is high, and packet loss occurs. In this case, you need to collect the IP address and DNS server address of the client, the ping command screenshot, and the My Traceroute (MTR) screenshot. In this case, you need to collect the IP address and DNS server address of the client, the ping command screenshot, and the My Traceroute (MTR) screenshot. The issue in this case may be caused by the following reasons.
Invalid acceleration region
A user in the Chinese mainland is allocated to a POP outside the Chinese mainland, or a user outside the Chinese mainland is allocated to a POP in the Chinese mainland. Scenarios:
In this case, we recommend that you set the acceleration region to Global.
If you set the acceleration region to Chinese Mainland Only, even requests from users outside the Chinese mainland are redirected to POPs in the Chinese mainland.
If you set the acceleration region to Global (Excluding the Chinese Mainland), even requests from users in the Chinese mainland are redirected to POPs outside the Chinese mainland.
Incorrect client DNS setting
The user needs to change to use the DNS server of the corresponding ISP and location.
If a China Mobile (Guangdong) user uses the DNS server of China Unicom, requests from the user are redirected to POPs of China Unicom, and long-distance routing extends the network link.
If a user of China Mobile (Guangdong) uses the DNS server of China Mobile (Harbin), requests from the user are redirected to POPs of China Mobile (Harbin), and long-distance routing extends the network link.
If the acceleration region and DNS server are correct, the user is allocated to a proper POP, but the network quality is still poor. Then you need to collect traceroute and MTR information for further diagnosis.
Case 2: low cache hit ratio or frequent origin fetches
Alibaba Cloud CDN accelerates the delivery of static resources by caching the static resources on POPs that are nearest to the clients. When a user requests a resource, the POP returns the cached resource to the user. This prevents origin fetch over a long link. A low cache hit ratio increases the workload on origin servers and causes low access efficiency for static resources. Therefore, the cache hit ratio affects user experience. It is important to ensure a high cache hit ratio. You can optimize the cache hit ratio based on the causes contributing to the low hit ratio. For more information, see Increase the cache hit ratios of Alibaba Cloud CDN. You can determine whether the cache is hit based on the X-Cache header in the response.
X-Cache: If the value of this field is MISS, the cache is missed and the request is redirected to the origin server. If the value of this field is HIT, the cache is hit and the cached resource is returned.
X-Swift-CacheTime: specifies the time to live (TTL) value of a resource that is cached on POPs. If the value is 0, resources cannot be cached.
Solutions for a low cache hit ratio or frequent origin fetches
The first access to an accelerated resource is slower than the direct access to the resource from the origin server, because the resource is not cached on POPs for the first access and needs to be retrieved from the origin server. In this case, we recommend that you use the URL-based prefetch feature to prefetch the content from the origin server to POPs. Then, the first time users access the prefetched content, the cache is hit. This accelerates content loading. For more information, see Purge and prefetch resources.
If a resource is infrequently accessed, Alibaba Cloud CDN receives only a small number of requests for these resources, and they do not hit the cache. POPs are shared by all Alibaba Cloud CDN users. Cache rules in Alibaba Cloud CDN specify the maximum time-to-live (TTL) of resources on POPs. If the traffic of your Alibaba Cloud CDN-accelerated domain name is low, your resources may be removed from the cache of POPs before they expire, The least popular cached resources are the first to be stale in the cache. Popularity is indicated by how frequently a resource is accessed on POPs. If the resource has low popularity, the resource may be removed before it expires.
An improper cache TTL is configured. Scenarios involving frequent origin fetches:
If you do not configure cache rules in Alibaba Cloud CDN and responses for static files do not contain the ETag or Last-modified headers, static files cannot be cached on POPs. To resolve this issue, you need to configure the response headers on the origin servers or configure cache rules in Alibaba Cloud CDN. For more information, see Cache settings.
If you do not configure a cache rule in Alibaba Cloud CDN, the default cache policy is used. The maximum cache TTL is 3,600 seconds. This may cause frequent origin fetches after the cached resources expire. We recommend that you set a proper cache TTL based on your business requirements. For more information, see Default cache rule and priorities of cache rules.
Cache-Control directives on the origin have a higher priority. Even if you configure a cache rule, Alibaba Cloud CDN does not cache resources as long as the origin response contains Cache-Control directives that do not allow caching. If one of the maxage=0, max-age=0, no-cache, no-store, private, and Pragma: no-cache rules exists in cache rules of the origin server, Alibaba Cloud CDN does not cache resources. In this case, you need to modify the rules on the origin server. Change the value to Public, which allows caching. For more information, see Configure an NGINX cache policy and Configure an Apache cache policy.
URLs containing variable parameters URLs for accessing resources contain variable parameters. When a client uses different URLs to access Alibaba Cloud CDN, Alibaba Cloud CDN considers that two different requests are initiated, and redirects the requests to the origin server to retrieve the file, even if the URLs point to the same file that is cached on POPs. In this case, we recommend that you enable the parameter filtering feature. For more information, see Ignore parameters.
Range origin fetch for large files If you want to cache large files, we recommend that you enable the range origin fetch feature. For more information, see Range origin fetch.
Case 3: slow dynamic requests
If clients access a dynamic resource, every request is made to the origin server to retrieve the resource and then return the resource to clients. In this scenario, Alibaba Cloud CDN cannot cache dynamic content that changes in real time. Therefore, Alibaba Cloud CDN is not suitable for accelerating dynamic content. POPs redirect requests for dynamic content to origin servers. Therefore, delivery of dynamic content is not accelerated. If your website or app contains a large amount of dynamic content, such as API operations, you can use the following solutions:
Separate static and dynamic resources. Use Alibaba Cloud CDN to accelerate static resources. Redirect dynamic requests to another domain name that is directly resolved to the origin server.
Use Dynamic Content Delivery Network (DCDN) to accelerate dynamic requests. For more information, see What is DCDN? Note: DCDN accelerates dynamic requests by using dynamic acceleration technologies provided by Alibaba Cloud, such as route optimization and transmission optimization. This allows you to retrieve resources from the origin server as fast as possible. DCDN optimizes the network layer. If the response of the origin server is slow, you need to optimize the origin server.
Case 4: slow response of the origin server
Requests for resources that are not cached and dynamic requests are redirected to the origin server. If the response of the origin server is slow, the overall response speed is slow. In this case, you can map a local domain name to the origin server or test the response speed of the origin domain. Causes:
The processing performance of the origin server is limited. For example, the bandwidth or CPU of the origin server reaches a bottleneck, or the processing speed of the origin server is slow. In this case, you can optimize the origin server. If the performance is poor, you need to scale out the origin server.
The network of the origin server is poor or the origin server involves cross-border connections. For example, a user requests a resource from a POP in the Chinese mainland, but the origin server is outside the Chinese mainland. Alibaba Cloud CDN retrieves content from the origin server over the Internet. Therefore, if Alibaba Cloud CDN retrieves content from the origin server over a cross-border connection, content retrieval may be affected. This is because cross-border connections involve different ISPs in and outside the Chinese mainland, and use the international Internet egress. In this case, the networks of Alibaba Cloud CDN and the origin server cannot be controlled, and Alibaba Cloud CDN has a limited potential for optimization. We recommend that you deploy one origin server for requests from the Chinese mainland and one origin server for requests from outside the Chinese mainland.
Case 5: slow loading of website homepages
When a client accesses http://www.example.com/, the browser requests the homepage. After the request is successful, the server returns HTML code to the browser. Then, the browser requests resources to be introduced into the code, such as images, JavaScript files, and CSS files, based on the returned HTML code. If the homepage is a dynamic resource or a resource that does not need to be cached, Alibaba Cloud CDN retrieves the resource from the origin server for each request for the homepage. A slow response from the origin server causes slow loading of the homepage. As a result, requests remain in the Pending state for a long time. For information about whether the cache is hit, see Case 2: low cache hit ratio or frequent origin fetches.
In scenarios where the homepage is not cached and take a long time to load, requests for the homepage remain in the Pending state for a long time. When the homepage is finally loaded, static resources are quickly loaded.
Case 6: large resource loading
If your website has a large amount of resources to be loaded, you can configure the performance optimization feature for the accelerated domain name. For more information, see Performance optimization overview. Intelligent compression supports the following formats: text/html, text/xml, text/plain, text/css, application/javascript, application/x-javascript, application/rss+xml, text/javascript, image/tiff, image/svg+xml, application/json, and application/xmltext.
Case 7: slow access by users of a specific ISP in a specific region
In some scenarios, clients have common problems. For example, a large number of China Mobile users in a specific city may report that access is slow or abnormal during a period of time, but access by China Unicom users is normal. This issue may be related to the network of the ISP or requested POPs in the region. In most cases, you can troubleshoot this issue by collecting ping information on the client and determine the network latency between the client and the POP.
In addition, you can bind the requested IP address to the POP for testing. The testing method is similar to testing on the origin server, except you need to change the IP address of the origin server to the IP address of the POP. Before you bind a POP, you can check whether the POP has a slow response speed. If the response speed is slow, check whether the request hits the cache and whether the loaded resources are too large. Then, analyze the issue based on the preceding cases. If you cannot identify the issue, submit a ticket to contact Alibaba Cloud.
Applicable scope
Alibaba Cloud CDN
DCDN