Automated threats — from credential stuffing and scanning tools to sophisticated fake crawlers — can degrade service performance, exhaust rate limits, and compromise user accounts. Bot threat intelligence rules let you apply targeted actions to requests from known malicious sources using the Alibaba Cloud crawler library, which covers more than 700 crawler types and is updated in real time based on threat intelligence and traffic analysis across Alibaba Cloud.
Six intelligence libraries are available, ranging from scanner fingerprint detection to broad IP block lists covering public clouds and on-premises data centers. Each library defaults to Monitor (log only). Customize the protected paths and response action to match your risk tolerance.
Prerequisites
Before you begin, make sure you have:
You have purchased a WAF instance in a region outside the Chinese mainland, and the Bot Management module is enabled for the instance.
A WAF instance with the Bot Manager feature enabled
Your website added to WAF (see Tutorial)
Before you enable a library
Review these notes before activating any intelligence library:
All traffic is inspected. After you enable bot threat intelligence, every request to your website is checked. To exempt specific requests, configure a bot management allowlist. See Configure a whitelist for Bot Management.
Fake Crawler Blacklist requires an allowlist first. This library blocks requests that use the User-Agent of authorized search engines (such as BaiduSpider) to impersonate legitimate crawlers. Without a crawler allowlist, legitimate search engine traffic may be blocked. See Configure the allowed crawlers function.
Malicious Crawler Blacklist has tiered false positive risk. Higher severity levels include more IP addresses and have a higher false positive rate. For the high-severity library, use both CAPTCHA and JavaScript validation to challenge rather than block requests outright. If challenge-based verification is not feasible, enable only the low-severity library.
Slider CAPTCHA supports synchronous requests only. The Captcha and Strict Captcha actions use slider CAPTCHA. Ajax and other asynchronous requests are not supported. To verify async traffic, contact the Alibaba Cloud security team or test with a custom protection policy. See Configure a custom protection policy.
Enable bot threat intelligence rules
Log on to the WAF console.
In the top navigation bar, select the resource group and the region where your WAF instance is deployed. You can select Chinese Mainland or Outside Chinese Mainland.
In the left-side navigation pane, choose Protection Configurations > Website Protection.
At the top of the Website Protection page, select the domain name you want to protect from the Switch Domain Name drop-down list.
Click the Bot Management tab, find the Bot Threat Intelligence section, turn on Status, and click Settings.
In the Bot Threat Intelligence rule list, find the library you want to activate and turn on the switch in the Status column. The following table describes each intelligence library. After you enable a library, WAF applies the Monitor action to matching requests — allowing the traffic through while recording it in logs.
Intelligence library Coverage Default action Malicious Scanner Fingerprint Blacklist Characteristics of tens of thousands of scanners, based on traffic analysis Monitor Malicious Scanner IP Blacklist IP addresses of scan attack sources, dynamically updated from Alibaba Cloud traffic Monitor Credential Stuffing IP Blacklist Hundreds of thousands of IP addresses associated with credential stuffing and brute-force attacks Monitor Fake Crawler Blacklist Crawlers that use the User-Agent of authorized search engines (such as BaiduSpider) to impersonate legitimate programs. ImportantConfigure a crawler allowlist before enabling this library to avoid false positives.
Monitor Malicious Crawler Blacklist Millions of malicious IP addresses, dynamically updated from crawler attack sources on Alibaba Cloud. Three severity levels are available: low, medium, and high. Higher severity includes more IP addresses but also increases the false positive rate. Monitor IDC IP Lists IP addresses of public clouds and on-premises data centers, including Alibaba Cloud, Tencent Cloud, Meituan Open Services, and 21Vianet. Attackers commonly route crawlers through these networks; regular users rarely access websites from them. Monitor (Optional) Customize a rule's protected path or action. The default rules apply to all directories (
/) of the domain name. To restrict protection to specific paths or change the response action, click Edit in the Actions column of the rule you want to modify. In the Edit Intelligence dialog box, configure the following parameters:Parameter Description Protected Path The URL path to protect, such as /loginor/api. Use/to cover all directories. For each path, select a matching mode: Precise Match (exact URL only), Prefix Match (URL starts with the specified path), or Regular Expression Match (URL matches a regular expression). Click Add Protected URL to add more paths. Up to 10 paths are supported.Action The action to take when a request matches the rule. Options: Monitor (allow and log), Block (deny the request), JavaScript Validation (challenge the client with a JavaScript check before allowing the request), Captcha (challenge the client with a slider CAPTCHA before allowing the request), or Strict Captcha (same as Captcha but with a stricter identity verification standard). Click OK to save.
What's next
To exempt specific requests from bot threat intelligence checks, see Configure a whitelist for Bot Management.
To allow known legitimate crawlers, see Configure the allowed crawlers function.
To test CAPTCHA-based rules on async traffic, see Configure a custom protection policy.