DCDN Referer whitelist and blacklist for hotlink protection - Edge Security Acceleration

A Referer blacklist or whitelist controls access to your resources based on the Referer header of HTTP requests. This feature helps prevent hotlinking and unauthorized use of your resources. After you configure this feature, DCDN evaluates the Referer header of incoming requests to allow or deny access.

Important

By default, the Referer blacklist/whitelist feature for Alibaba Cloud DCDN is disabled, which means any website can access your resources.
Using a Referer blacklist or whitelist is one way to prevent hotlinking. For more methods, see Best practices for preventing hotlinking.
After you add a domain name to the Referer blacklist or whitelist, DCDN adds the wildcard version of that domain to the list. For example, if you enter aliyundoc.com, the rule for *.aliyundoc.com takes effect, which means that all of its subdomains are affected.

Use cases

Referer blacklists and whitelists are primarily used to protect your website resources from hotlinking by other websites. Common use cases include:

Copyright protection: For websites with copyrighted content, you can use a Referer blacklist or whitelist to restrict access to authorized websites only, thereby protecting your copyrights.
Hotlinking prevention: A Referer blacklist or whitelist ensures that your resources can be used only on specific websites. This prevents other sites from directly linking to your content and reduces unauthorized bandwidth consumption.
Enhanced website security: By allowing only specific websites to access your resources, you can prevent malicious hotlinking, unauthorized access, and theft of sensitive information.
Traffic source control: A Referer blacklist or whitelist allows you to restrict traffic from specific websites, which helps you control traffic sources and improve the stability and security of your website.

In summary, the Referer blacklist/whitelist feature is a versatile tool for protecting resources, enhancing security, and controlling traffic.

How it works

A POP checks the Referer header of each request. If a request's Referer is in the blacklist, or not in the whitelist, the POP denies the request, saving bandwidth. The Referer request evaluation rules:

If the Referer in a request matches an entry in the blacklist, or does not match any entry in the whitelist, DCDN denies the request.
If the Referer in a request matches an entry in the whitelist, DCDN allows the request.

Procedure

Log on to the DCDN console.
In the left-side navigation pane, click Domain Names.
On the Domain Names page, find the target domain name and click Configure.
In the left-side navigation tree of the domain name, click Access Control.
On the Hotlink Protection tab, turn on Hotlink Protection.

Select Blacklist or Whitelist based on your needs.

Configure Referer-based hotlink protection

Parameter		Description
Type		Blacklist Requests from blacklisted domains are denied. Whitelist Only requests from whitelisted domains can access your resources. Note The blacklist and whitelist are mutually exclusive.
Rules		Enter one domain name per line. No leading spaces. Asterisks () work as wildcards. For example, `.developer.aliyundoc.com` matches `image.developer.aliyundoc.com` and `video.developer.aliyundoc.com`. Note Maximum rule size: 60 KB.
Redirect URL		Blocked requests receive an HTTP 302 redirect to this URL. The URL must start with `http://` or `https://`, for example, `http://www.example.com`.
Advanced Settings	Allow resource URL access from browsers	Disabled by default. When enabled, requests with an empty Referer header can access resources regardless of whether you configure a whitelist or blacklist. An empty Referer occurs when: No Referer header is present. The Referer header value is empty.
	Exact Match	Not selected by default. If you select this check box, subdomains cannot be matched. For example, adding `example.com` matches only `example.com` — subdomains are excluded.
	Ignore Scheme	When disabled, the Referer value must include an `http://` or `https://` scheme. When enabled, the Referer value is matched without requiring an `http://` or `https://` scheme.

Click OK.

Matching logic

The following table shows how DCDN matches the Referer header against your list. Requests that match a blacklist entry or fail to match any whitelist entry are rejected by DCDN with HTTP 403.

Configured domain name	Referer header value in a request	Matched	Description
www.example.com *.example.com	http://www.example.com/img.jpg	Yes	Referer domain matches a list entry.
	http://www.example.com:80/img.jpg	Yes	Referer domain matches a list entry.
	www.example.com	No	No HTTP/HTTPS scheme in the Referer value.
	http://aaa.example.com	Yes	Wildcard *.example.com matches subdomains at any depth.
	http://aaa.bbb.example.com	Yes	Wildcard *.example.com matches subdomains at any depth.
	http://example.com	No	*.example.com matches subdomains but not the root domain.
	http://www.example.net	No rules matched	Domain not in list; default allow rule applies.

FAQ

Why are schemes missing from Referer headers?

In most cases, the Referer header in a user request includes the HTTP or HTTPS scheme. However, in certain situations, the scheme may be missing.

A common scenario is when a user navigates from a non-secure website (HTTP) to a secure website (HTTPS). Browsers may modify or remove parts of the Referer header based on security policies, such as the Referrer-Policy, to protect user data. In this case, the Referer header might only contain the domain name without the scheme.

Additionally, some browsers or proxy servers may automatically remove the Referer header in specific situations, such as when using a privacy mode or accessing a website through an anonymous proxy.

Therefore, it is important to handle cases where the Referer header might not include the scheme to ensure correct evaluation.

Empty Referer headers: causes and handling

An empty Referer occurs when the Referer request header is either missing from an HTTP request or its value is empty. Normally, the Referer header contains a complete URI, including the scheme (such as http or https), hostname, and possibly the path and query string. An empty Referer may occur for several reasons:

Direct access: When a user enters a URL directly into the browser's address bar, uses a bookmark, or opens a link in a new tab, there is no referring page, so the Referer header is empty.
User privacy settings: Users or their software (such as browser extensions or privacy modes) may intentionally remove the Referer header to protect privacy.
Security protocols: When a user navigates from an HTTPS page to an HTTP page, browsers typically do not send the Referer header to prevent potential leakage of sensitive information.
Client-side policies: Some websites or applications may control the sending of the Referer header for security reasons by setting a <meta> tag or an HTTP header like Referrer-Policy.
Cross-origin requests: Some cross-origin requests may not include the Referer header due to browser security policies.

How you handle requests with an empty Referer depends on your specific application and security requirements. Here are some recommendations:

Default policy: If your service does not depend on Referer information to make decisions, you can allow requests with an empty Referer.
Allow access for specific URLs: You can select the Allow resource URL access from browsers option. This allows requests even if the Referer is empty, allowing POPs to grant access to the resource.