Prevent malicious traffic with Referer rules - CDN - Alibaba Cloud Documentation Center

A Referer blacklist or whitelist controls access to your resources based on the Referer header of HTTP requests. This feature prevents hotlinking and unauthorized resource use. Once configured, CDN and evaluate the Referer header to grant or deny access.

Important

By default, the Referer blacklist/whitelist feature for Alibaba Cloud CDN and is disabled, allowing any website to access your resources.
Referer blacklists and whitelists are one way to prevent hotlinking. For more methods, see Best practices for preventing hotlinking.
When you add a domain name like aliyundoc.com to a Referer list, CDN and apply the rule to both the domain itself and all its subdomains (such as *.aliyundoc.com).

Anatomy of a Referer

The Referer header indicates the URL of the page where a request originated. A Referer URL consists of a scheme, domain, path, and query parameters, as shown in the following figure.

Note

The value of a Referer header is essentially a URL.
Alibaba Cloud supports domain-only Referer configurations. To enable this, select the Ignore Scheme option.

Use cases

A Referer blacklist or whitelist primarily protects your website resources from being directly linked or used by other websites. Common use cases include:

Copyright protection: For websites with copyrighted content, you can use a Referer whitelist to restrict access to authorized websites only.
Hotlinking prevention: A whitelist ensures that your resources can be accessed only from specific websites, preventing other sites from consuming your bandwidth by directly linking to your content.
Enhanced website security: By allowing only specific websites to access your resources, you can prevent malicious hotlinking, unauthorized access, and theft of sensitive information.
Traffic source control: You can restrict traffic from specific websites to control traffic sources and improve the stability and security of your website.

How it works

A Point of Presence (POP) checks the Referer header of each request. If the Referer is not on the whitelist or if it is on the blacklist, the POP denies the request. This saves bandwidth and reduces load on the origin server. CDN and use the following rules to process Referer requests:

If the Referer in a request matches an entry in the blacklist or does not match any entry in the whitelist, CDN and deny the request.
If the Referer in a request matches an entry in the whitelist, CDN and allow the request.

Usage notes

After you configure a Referer blacklist, requests from blacklisted sources can still reach CDN and POPs. However, these requests are rejected with a 403 status code and are recorded in CDN and logs.
The Referer blacklist/whitelist feature enforces access control based on the Referer header in an HTTP request. Blocking requests incurs a small fee for the generated traffic. For HTTPS, you are also charged per request.
Because CDN hotlinking originates from public network access, Referer rules apply only to public domain names.

Procedure

Log on to the CDN console.
In the left navigation pane, click Domain Names.
On the Domain Names page, find the target domain name and click Manage in the Actions column.
In the domain's navigation pane, click Access Control.
On the Referer Black/Whitelist tab, click Modify.
Configure the Referer parameters. For more information, see Referer configuration parameters.
Click OK to save the configuration.

Referer configuration parameters

Parameter		Description
Type		Blacklist The POP denies access to requests that contain a Referer from the blacklist. Whitelist The POP allows only requests that contain a Referer from the whitelist. Note Blacklists and whitelists are mutually exclusive. You can select only one type at a time.
Rules		You can add multiple Referers. Separate each entry with a line break. You can use an asterisk () as a wildcard to match all subdomains. For example, `.example.com` matches all subdomains of `example.com`. If you do not specify a wildcard, an entry matches both the domain itself and all its subdomains. For example, an entry for `example.com` matches both `example.com` and `.example.com`. Note* The total size of all rules in a Referer list cannot exceed 60 KB. You do not need to include the scheme in the rules.
Redirect URL		If a request is blocked, the POP returns a 302 redirect with a Location response header. This parameter specifies the value for the Location header. The URL must start with `http://` or `https://`. Example: `http://www.example.com`.
Advanced Settings	Allow resource URL access from browsers	This option is cleared by default. If you select this option, requests with an empty referer are not blocked and are allowed to access the resource, regardless of your list settings. An empty referer is defined as either of the following: The request does not include a Referer header. The Referer header value is empty.
	Exact Match	This option is cleared by default. If you select this option, the default behavior of matching subdomains is disabled. If you do not use a wildcard, an entry such as `example.com` matches only `example.com`.
	Ignore Scheme	If the Ignore Scheme option is cleared, the Referer header must include an HTTP or HTTPS scheme. If you select the Ignore Scheme option, the Referer header is not required to include a scheme.
Rule Condition		A rule condition allows a rule to be applied only when a request meets specific criteria. Important When a feature references rule conditions, the execution order follows the priority of the associated rule conditions, not the order of the feature configurations. Do not use: Disables conditional rules. You can add or edit conditional rules in the Rules engine.

Configuration verification

You can use the curl command to test your configuration. Use the -e option to specify the referer value and the -I option to retrieve only the response headers of the accelerated domain name. The following examples use a whitelist for verification.

Scenario 1: Rules parameter only

Note

In this scenario, a Referer rule for aliyun.com is configured. The Redirect URL, Advanced Settings, and Rule Condition parameters are not configured.
This configuration matches requests that contain a Referer from http(s)://aliyun.com or its subdomains. Requests with Referers that are not on the whitelist are rejected.

Test access with a Referer from the main domain. Command: curl -e http://aliyun.com -I <accelerated_domain_name>.
```
HTTP/1.1 200 OK
Server: Tengine
```
Test access with a Referer from a subdomain. Command: curl -e http://sub.aliyun.com -I <accelerated_domain_name>.
```
HTTP/1.1 200 OK
Server: Tengine
```
Test access with a Referer from a different domain. Command: curl -e http://aIiyun.com -I <accelerated_domain_name>.
```
HTTP/1.1 403 Forbidden
Server: Tengine
```
Test access with an empty Referer. Command: curl -e "" -I <accelerated_domain_name>.
```
HTTP/1.1 403 Forbidden
Server: Tengine
```
Test access with a domain-only Referer. Command: curl -e aliyun.com -I <accelerated_domain_name>.
```
HTTP/1.1 403 Forbidden
Server: Tengine
```

Scenario 2: Rule and Allow resource URL access from browsers

Note

In this scenario, the Referer rule is set to aliyun.com and the Allow direct access to resource URLs from browsers option is selected. Other advanced settings, the redirect URL, and the rule condition are not configured.
Compared with Scenario 1, this configuration also allows requests with an empty Referer, such as requests from direct browser access.

Test access without a Referer. Command: curl -I <accelerated_domain_name>.
```
HTTP/1.1 200 OK
Server: Tengine
```
Test access with a blank Referer value (""). Command: curl -e "" -I <accelerated_domain_name>.
```
HTTP/1.1 200 OK
Server: Tengine
```

Scenario 3: Rule and Ignore Scheme

Note

In this scenario, the Referer rule is set to aliyun.com and the Ignore Scheme option is selected. Other advanced settings, the redirect URL, and the rule condition are not configured.
This configuration is similar to Scenario 1, but it also allows Referer requests that do not include a scheme, such as a request with only the domain name aliyun.com.

Test access with a Referer that does not have a scheme. Command: curl -e aliyun.com -I <accelerated_domain_name>.

HTTP/1.1 200 OK
Server: Tengine

FAQ

Missing Referer scheme

In most cases, the Referer header in a user request includes an HTTP or HTTPS scheme. However, in certain situations, the Referer may not include a scheme.

A common scenario is when a user navigates from a non-secure website (which uses HTTP) to a secure website (which uses HTTPS). Browsers may modify or remove the Referer header based on security policies, such as the Referrer-Policy, to protect user data. In this case, the Referer header might contain only the domain, without the scheme.

In addition, some browsers or proxy servers may automatically remove the Referer header in specific situations, such as when private browsing mode is used or a website is accessed through an anonymous proxy.

Therefore, you must account for requests that lack a scheme in the Referer header. If you want to match such requests, select the Ignore Scheme option.

Handling empty Referer

An empty Referer occurs when the Referer header is missing from an HTTP request or its value is empty. The Referer header in a request usually contains a complete URI that includes the scheme (such as http or https), a hostname, and possibly a path and query string. An empty referer can occur in the following scenarios:

Direct access: When a user enters a URL directly into the browser's address bar, uses a bookmark, or opens a link in a new blank tab, no referring page exists. Therefore, the Referer header is empty.
User privacy: A user or software used by the user, such as a browser extension or a privacy mode, may intentionally remove the Referer header to protect privacy.
Security protocol: When a user navigates from an HTTPS page to an HTTP page, the browser usually does not send the Referer header to prevent the leakage of sensitive information.
Client-side policy: Some websites or applications may control whether the Referer header is sent for security reasons by setting a <meta> tag or an HTTP header such as Referrer-Policy.
Cross-origin request: Some cross-origin requests may not include the Referer header due to browser security policies.

How you handle requests with an empty Referer depends on your specific use case and security requirements. The following are some recommendations:

Default policy: If your service does not rely on Referer information to make decisions, you can allow requests with an empty Referer.
Allow access: For specific URLs or origins, you can select the Allow resource URL access from browsers option. This allows access even if the Referer is empty. In this case, CDN and POPs grant access to the resource.