WAF's Bot Management module lets you configure scenario-specific anti-crawler rules to protect native iOS and Android apps from malicious crawlers. Rules apply signature verification, device behavior checks, and rate limiting to traffic you define—scoped to a specific business flow such as login, registration, or order placement.
Test every rule against a single IP address in the Verify Actions step before publishing. Skipping this step risks blocking legitimate users if the rule is misconfigured or if SDK compatibility issues exist.
How it works
When a request matches your configured traffic characteristics, WAF evaluates it against three protection layers: SDK signature verification, device behavior analysis, and rate limiting. The Anti-Bot SDK embedded in your app adds a signature to each request, which WAF uses to identify and handle crawler traffic more precisely. You can also use intelligent algorithms together with the configured anti-crawler rules to automatically handle matching traffic.
The verification step (step 7) scopes the rule to a single test IP, so you can confirm the rule works correctly before it applies to all production traffic.
Prerequisites
Before you begin, ensure that you have:
A subscription WAF instance running the Pro, Business, or Enterprise edition with the Bot Management module enabled
Your website added to WAF (see Tutorial)
Anti-Bot SDK integrated into the apps you want to protect (see Integrate Anti-Bot SDK into apps)
Create an anti-crawler rule for apps
The configuration wizard has four steps: define the scenario, configure protection rules, verify the rule, and publish.
Log on to the WAF console. In the top navigation bar, select the resource group and the region where your WAF instance is deployed (Chinese Mainland or Outside Chinese Mainland).
In the left-side navigation pane, choose Protection Configurations > Website Protection.
In the upper part of the Website Protection page, select the domain name you want to protect from the Switch Domain Name drop-down list.
On the Bot Management tab, start the rule wizard:
First rule: Click Start in the Scenario-specific Configuration section.
Additional rules: Click Add in the upper-right corner.
Each domain name supports up to 50 anti-crawler rules.
In the Configure Scenarios step, set the scenario parameters and click Next.
Parameter Description Scenario The type of business flow to protect, such as logon, registration, or order placement. Service Type Select Apps to protect native iOS and Android apps. For HTML5 apps, select Websites instead. Traffic characteristics Match conditions that identify traffic destined for your apps. Each condition specifies a matching field (an HTTP request header field), a logical operator, and matching content. You can add up to five conditions. If you enter an IP address as matching content, press Enter to confirm. For available fields, see Fields in match conditions. In the Configure Protection Rules step, configure the anti-crawler rules and click Next.
Parameter Description Check Invalid App Signature Always enabled. WAF detects requests with invalid or missing SDK signatures. Set Action to Monitor (log only) or Block (reject the request). Check Abnormal Device Behavior When enabled, WAF flags requests from devices with suspicious characteristics: Use Simulators, Use Proxies, Use Rooted Device, Debugging Mode, Hooking, or Multiboxing (running multiple instances of a protected app simultaneously). Set Action to Monitor or Block. Action Applies to both Check Invalid App Signature and Check Abnormal Device Behavior. Monitor logs matching requests without blocking them. Block rejects matching requests. IP Address Throttling When enabled, WAF monitors the request rate per IP address. If requests from an IP address exceed the threshold within the specified time window, WAF applies the Monitor or Block action to subsequent requests from that IP for the configured duration. You can define up to three conditions. For details, see Configure a custom protection policy. Device Throttling When enabled, WAF monitors the request rate per device. If requests from the same device exceed the threshold within the specified time window, WAF applies the Monitor or Block action for the configured duration. You can define up to three conditions. Custom Session-based Throttling When enabled, WAF monitors the request rate per session. If requests from the same session exceed the threshold within the specified time window, WAF applies the Monitor or Block action for the configured duration. You can define up to three conditions. For details, see Configure a custom protection policy. In the Verify Actions step, test the rule before publishing. This step limits rule enforcement to a single test device so your production traffic is unaffected. Complete all three sub-steps: After the test, click I Have Completed Test to proceed. If the result shows issues, click Go Back to adjust the rule and retest. To skip this step, click Skip in the lower-left corner.
Enter a public IP address: Enter the public IP address of your test device (for example, a mobile phone). During the test, the rule applies only to requests from this IP address. > Tip: To find your public IP address, click Alibaba Network Diagnose Tool and look up Local IP, or search for your IP address in a browser.
Verify the SDK signature: Click Start Test to confirm that the Anti-Bot SDK is correctly integrated and generating valid signatures. > Note: Anti-Bot SDK must be installed on the test device. If it is not, the signature check fails and the test cannot complete.
Select an action: Check whether the Block action is in effect. After clicking Start Test, WAF immediately delivers the rule to your test device and displays the expected test procedure and result. Follow the on-screen instructions to verify that the Block action works correctly.
In the Preview and Publish Protection Rules step, review the rule and click Publish. The rule takes effect immediately.
If this is the first anti-crawler rule you have published for this domain, the rule ID is not displayed until after publishing. Find the rule ID on the Bot Management tab of the Security Report page. Use the rule ID to filter matching requests in Log Service for WAF.
FAQ
No valid test requests detected
The test request did not reach WAF or does not match the rule. Check the following:
Confirm the test request is sent to the IP address that maps to the CNAME record provided by WAF.
Compare the HTTP header fields in the test request against the Traffic Characteristics you configured. If they do not match, update the traffic characteristics.
Confirm that the public IP address of your test device matches the IP address you entered in step 7. Use the Alibaba Network Diagnose Tool to get the correct IP.
Test requests failed verification
WAF detected that the test did not simulate real user behavior. Debugging mode or automation tools can trigger this error. Use a real device and simulate actual user access to the app without enabling developer tools or proxy software during the test.
Also check:
Service type mismatch: If you selected Websites instead of Apps, change the Service Type parameter and retest.
Incorrect intermediate domain name: If your setup uses an intermediate domain name, select Use Intermediate Domain Name and choose the correct intermediate domain name from the list.
Frontend compatibility issue: If the error persists, submit a ticket to Alibaba Cloud technical support.
No verification triggered
No test rule was generated. Run the test several times until a test rule is generated, then proceed.