To configure anti-crawler rules for web applications, integrate the Anti-Bot SDK (SDK) into your web application first.
SDK components
The SDK includes two components: the web collector and the asynchronous API response component.
Web collector
The web collector embeds browser-side signals into the anti-bot system to detect attacks that network-level analysis alone cannot catch.
After collecting browser characteristics, the web collector obfuscates and encrypts the data, stores it as cookies on the root domain, and sends those cookies with each request. This approach minimizes impact on browser performance.
The web collector gathers the following data:
Browser information: browser type and version, screen resolution, time zone, and timestamp
Bot detection probes: signals specific to common browser-level bot scripts and drivers
User interaction events: keyboard, mouse, and touch events
Keyboard events capture only the timing of key presses, not which keys are pressed. This design protects user privacy.
Asynchronous API response component
The asynchronous API response component handles challenge responses—such as JavaScript validation and CAPTCHA verification—that WAF sends to your application's API endpoints.
The asynchronous API response component does not collect or report data and does not add security logic on its own. It only handles challenge responses issued by WAF.
How it works
The component globally rewrites common API request objects in the requested page—XMLHttpRequest (XHR), Fetch, and Form—and wraps each with an additional code layer. The original behavior of these objects is not affected.
For each API response, the component checks whether the response originates from WAF or from your origin server.
If the response is from your origin server: the component passes the response through to your page's JavaScript unchanged.
If the response is from WAF: the component parses the challenge response, performs the required JavaScript calculations, and sends a follow-up request containing the JavaScript validation signature back to WAF. WAF verifies the signature and forwards the request to your origin server.
Compatibility
| Requirement | Detail |
|---|---|
| Browser support | Browsers using the Internet Explorer 8 rendering engine or later |
| Cookie support | Client browsers must support cookies. If cookies are disabled, the web collector cannot function. |
| Hook compatibility | For services that hook native objects (XHR, Form, Fetch) in synchronous API requests, those hooks may conflict with the asynchronous API response component. |
Deploy the SDK
Three deployment methods are available. Choose based on whether your HTML page requests pass through WAF and whether you have Content Security Policy (CSP) restrictions.
| Method | When to use |
|---|---|
| Automatic integration | HTML page requests pass through WAF and your pages use gzip compression or no compression |
| Manual integration | HTML page requests do not pass through WAF, or your pages use Brotli or deflate compression |
| On-premises deployment | CSP blocks loading scripts from alicdn, or all components must be hosted on your own infrastructure |
Automatic integration
When HTML page requests pass through WAF, the bot management module parses each HTML response, injects the web collector and the asynchronous API response component into the Document Object Model (DOM), and returns the modified page to the client. No code changes to your HTML pages are required, and pages update at runtime without a full refresh.
Automatic integration supports only the content-encoding: gzip compression method. Brotli and deflate compression are not supported.For setup instructions, see Configure anti-crawler rules for websites.
Manual integration
Use manual integration when automatic integration is not available—for example, when HTML page requests do not pass through WAF, or when your pages use a compression method that automatic integration does not support.
Prerequisites
Before you begin, ensure that you have:
Identified the HTML pages where asynchronous calls are initiated (the SDK must be added to those pages)
Retrieved the SDK script from the WAF console
Get the SDK script
Go to the Bot Management page.
On the Scenario-specific Protection tab, click Create Template.
In the Configure Scenarios step, set Web SDK Integration to Manual Integration.
Click Obtain SDK to get the script.
The script takes the following form:
<script src="//o.alicdn.com/frontend-lib/common-lib/jquery.min.js"></script>Add the script to each HTML page where asynchronous calls are initiated.
On-premises deployment
Use on-premises deployment only when CSP prevents loading scripts from alicdn, or when your security policy requires all components to run on your own infrastructure.
To deploy the SDK on-premises:
Create two copies of the JavaScript resources in your on-premises environment.
Copy all JavaScript code from Alibaba Cloud CDN to your on-premises environment.
Place these copies before all other resources on the page to ensure they load first.
What's next
Integrating the SDK is the first step. To activate anti-crawler protection, configure scenario-specific protection templates to define how WAF responds to bot activity. See Configure anti-crawler rules for websites.