Configure a Referer whitelist or blacklist to protect websites from hotlinking - CDN

Referer-based hotlink protection refers to access control based on the Referer header. For example, you can configure a Referer whitelist to allow only specific requests to access your resources or a blacklist to block specific requests. Referer-based hotlink protection identifies and filters user identities and protects your resources from unauthorized access. After you configure a Referer whitelist or blacklist, Alibaba Cloud CDN allows or blocks requests based on user identities. If a request is allowed, Alibaba Cloud CDN returns the URL of the requested resource. Otherwise, Alibaba Cloud CDN returns HTTP status code 403.

Background information

The Referer header is a component of the header section in HTTP requests and contains information about the source address, including the protocol, domain name, and query string. The Referer header is used to identify the source of a request.

Referer-based hotlink protection is a server-side access control mechanism that is designed to protect resources from unauthorized access. When a user visits a website and clicks a link, the browser automatically adds a Referer field to the HTTP request header, which specifies the URL of the page from which the request is originated.

Important

By default, Referer-based hotlink protection is not enabled in Alibaba Cloud CDN. This means that all websites can access your resources.
After you add a domain name to the Referer whitelist or blacklist, the wildcard domain name that matches the domain name is automatically added to the whitelist or blacklist. For example, if you add aliyundoc.com to the Referer whitelist or blacklist, hotlink protection takes effect for all domain names that match *.aliyundoc.com.
After a Range request is initiated from a domain name, the browser adds the Referer header to the second Range request to identify the referring page of the request. To ensure that subsequent Range requests are not blocked by hotlink protection, add the domain name to the Referer whitelist.

Scenarios

A Referer whitelist or blacklist is suitable for the following scenarios:

Copyright protection: To safeguard copyrighted content on your website, you can use a Referer whitelist or blacklist to allow only authorized websites to access the content.
Hotlink protection: You can configure a Referer whitelist or blacklist to prevent your resources from being used by other websites.
Enhanced website security: Only websites that are included in a Referer whitelist that you configured are allowed to access your website resources. This prevents malicious hotlinking or theft of sensitive information.
Traffic source management: You can manage the domains that are authorized to use your resources. This ensures the security and stability of your website.

You can use the hotlink protection feature of Alibaba Cloud CDN in different scenarios to protect your website assets, manage traffic sources, and improve website security.

How it works

The server checks the Referer field of each request and rejects a request if the Referer field in the request does not match the pre-configured whitelist of trusted websites. This prevents other websites from directly linking to the resources of the website and helps save bandwidth and server resources. After you configure a Referer whitelist or blacklist, Alibaba Cloud CDN determines whether to allow a request based on the Referer header in the request and the Referer rules:

If the Referer header in the request is included in the Referer blacklist or is not included in the Referer whitelist, Alibaba Cloud CDN rejects the request.
If the Referer header in the request is included in the Referer whitelist, Alibaba Cloud CDN allows the request.

Procedure

Log on to the Alibaba Cloud CDN console.
In the left-side navigation pane, click Domain Names.
On the Domain Names page, find the domain name that you want to manage and click Manage in the Actions column.
In the left-side navigation tree of the domain name, click Access Control.
On the Hotlink Protection tab, click Modify.

Select Blacklist or Whitelist based on your business requirements.

Parameter	Description
Type	Blacklist Requests from domain names that are included in the blacklist cannot access your resources. Whitelist Only requests from domain names that are included in the whitelist can access your resources. Note The blacklist and whitelist are mutually exclusive. You can configure only one type of list at a time.
Rules	You can add multiple domain names to the Referer whitelist or blacklist. Enter one domain name per line. Do not add a space in front of the domain names. You can use asterisks () as wildcards. For example, if you add `.developer.aliyundoc.com` to the whitelist or blacklist, `image.developer.aliyundoc.com` or `video.developer.aliyundoc.com` can be matched. Note The content that you enter in the Rules field cannot exceed 60 KB.
Redirect URL	If a request is blocked, HTTP status code 302 and the Location header are returned instead of HTTP status code 403. This parameter is the value of the Location header. The value must start with `http://` or `https://`, such as, `http://www.example.com`.
Advanced Settings
Allow resource URL access from browsers	By default, the check box is not selected. If you select the check box, requests that contain an empty Referer header are allowed to access CDN resources, regardless of whether you configure a Referer whitelist or blacklist. An empty Referer header may suggest one of the following scenarios: The Referer header is not included in the requests. The Referer header is included in the requests, but the value is empty.
Exact Match	If Exact Match is not selected: By default, the check box is not selected. Fuzzy match is supported. Suffix match is supported. Do not add a period (.) to the left of a domain name because the system automatically adds a period (.) to the left of a domain name when the configuration is delivered. If you add `example.com` to the whitelist or blacklist, `example.com` and `<anyCharacter>.example.com` are matched. If you add `ab.example.com` to the whitelist or blacklist, `a<anyCharacter>b.example.com` and `<anyCharacter>.a<anyCharacter>b.example.com` are matched. If Exact Match* is selected: Exact match is supported, but subdomains cannot be matched. If you add `example.com` to the whitelist or blacklist, only `example.com` is matched. If you add `a*b.example.com` to the whitelist or blacklist, only `a<anyCharacter>b.example.com` is matched. Suffix match is not supported.
Ignore Scheme	Regardless of whether the Referer blacklist or whitelist is configured: If you do not select Ignore Scheme and the value of the Referer header does not start with HTTP or HTTPS, the Referer is considered invalid. For example, the `www.example.com` Referer is invalid. Only Referers in the `https://www..com` or `http://www..com` format are valid. If you select Ignore Scheme and the Referer header that is included in a request does not have an HTTP or HTTPS header, the Referer is considered valid. For example, the `www.example.com` Referer is valid.
Rule Condition	Rule conditions can identify parameters in a request to determine whether a configuration takes effect on the request. Do not use conditions Select the configured rule conditions in Rules Engine. For more information, see Rules engine.

Click OK.

Matching logic

The following table describes the matching logic of the Referer header. If the Referer header in a request does not match the whitelist or matches the blacklist, Alibaba Cloud CDN rejects the request and returns HTTP status code 403.

Configured domain name	Referer header value in a request	Matched?	Description
www.example.com *.example.com	http://www.example.com/img.jpg	Yes	The domain names in the Referer header match the domain names that are configured in the Referer whitelist or blacklist.
	http://www.example.com:80/img.jpg	Yes
	www.example.com	See the Description column.	If Ignore Scheme is not selected, the domain name is not matched because the HTTP or HTTPS header is not included in the Referer. If Ignore Scheme is selected, the domain name is matched.
	http://aaa.example.com	Yes	The domain name is matched regardless of whether you select Exact Match.
	http://aaa.bbb.example.com	See the Description column.	If Exact Match is not selected, the domain name is matched. This is because `.example.com` is configured in the whitelist or blacklist. In this case, subdomains in the Referer header are covered by the wildcard domain name. If Exact Match* is selected, the domain name is not matched because `*.example.com` is configured in the whitelist or blacklist. Only domain names of the level at which the wildcard is declared can be matched.
	http://example.com	No	The domain name in the Referer header does not match the wildcard domain name in the Referer whitelist or blacklist. This is because a wildcard domain matches subdomains but does not cover the root domain.
	http://www.example.net	No rules matched	The domain name in the Referer header is not included in the blacklist or whitelist. Therefore, the request is allowed according to the default rule.

FAQ

Why is the HTTP or HTTPS string occasionally missing in the Referer header in a request?

In most cases, the HTTP or HTTPS string is included in the Referer header in a request.

However, in some cases, for example, when a browser navigates a request from a website that does not use HTTPS to a website that uses HTTPS, the browser may present only the domain name in the Referer header to protect sensitive user data.

In addition, some browsers or proxy servers may automatically exclude the protocol string from the Referer header in specific scenarios, such as access in private browsing mode or by using an anonymous proxy.

Therefore, in actual practice, take note of the scenarios in which HTTP or HTTPS is not included in the Referer header when you configure hotlink protection. If you want to allow requests whose Referer header does not include HTTP or HTTPS, select Ignore Scheme.

Why is the Referer header empty in a request? What do I do to resolve the issue?

In most cases, the Referer header in a request contains the full URI, which includes the protocol, such as http or https, the hostname, and possibly the path and query string. The Referer header in a request may be empty due to the following reasons:

Direct access: If a user enters a URL in the address bar of a browser, uses a bookmark, or opens a new blank browser tab, the Referer header is empty because a referring page does not exist.
User privacy settings: Users configure private browsing mode or use privacy-focused extensions to remove the Referer header out of privacy concerns.
Security protocol: If a request is redirected from an HTTPS page to an HTTP page, the browser does not present the Referer header to prevent leakage of sensitive information.
Client policy: For security purposes, some websites or applications may restrict the browser from sending the Referer header by specifying the <meta> tag or HTTP headers, such as Referrer-Policy.
Cross-origin requests: Specific cross-origin requests may not include the Referer header based on the security policy of the browser.

The handling measures vary with different scenarios and security requirements:

Default policy: If your service does not rely on the Referer header, you can allow requests that have an empty Referer header.
Allow access: For specific URLs or sources, you can select Allow resource URL access from browsers to allow only requests from these URLs or sources. This way, Alibaba Cloud CDN points of presence (POPs) allow users to access your resources regardless of whether the Referer header is empty.