All Products
Search
Document Center

Web Application Firewall:Enable and configure the bot management module

Last Updated:Feb 22, 2024

The bot management module of Web Application Firewall (WAF) can be used to mitigate attacks that are initiated by automation tools such as emulators and malicious scripts. The attacks include data crawling, fraud, credential stuffing attacks, spam user registrations, auto-purchase bots, promotion abuse, and SMS flood attacks. We recommend that you configure bot management rules based on analytical data of bot traffic to reduce data leaks and risks during marketing activities. This helps reduce server workloads and bandwidth costs. This topic describes how to enable the bot management module and configure bot management policies.

Feature description

The bot management module provides bot traffic analysis, basic protection, and scenario-specific protection. The bot management module helps you identify bot traffic and defend against crawlers to protect your web services from being crawled.

  • Bot Traffic Analysis You can use the feature regardless of whether you enable the bot management module.

    You can view analytical data of bot traffic without enabling the bot management module, including bot traffic trends, top 20 risky clients, top 20 risky IP addresses, and analytical data of bot traffic to protected objects. This helps you identify and locate risky endpoints in an efficient manner. For more information, see View analytical data of bot traffic.

    You can apply for a trial or enable the bot management module to configure scenario-specific protection. For information about how to apply for a trial and how to enable the bot management module, see Enable the bot management module.

  • Scenario-specific Protection You must enable the bot management module before you can use the feature.

    The scenario-specific protection feature provides SDKs that you can integrate to configure custom rules to protect your websites and apps from crawlers. For more information, see Create an anti-crawler rule for websites and Create an anti-crawler rule for apps.

    This feature is suitable for users who are sensitive to bot traffic.

  • Basic Protection You must enable the bot management module before you can use the feature.

    The basic protection feature detects Layer 4 and Layer 7 bot traffic by using fingerprinting techniques. You can use the feature without the need to integrate SDKs. For more information, see Create a basic protection rule.

    This feature is suitable for users who want to defend against low-level crawlers by configuring protection policies in a simplified manner.

Prerequisites

View analytical data of bot traffic

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and the region in which the WAF instance is deployed. The region can be Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Scenario-specific Protection > Bot Management.

  3. On the Bot Traffic Analysis tab, view bot traffic trend, top 20 risky clients, top 20 risky IP addresses, and analytical data of bot traffic to protected objects. image.png

Enable the bot management module

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and the region in which the WAF instance is deployed. The region can be Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Scenario-specific Protection > Bot Management.

  3. Enable the bot management module.

    • Apply for a free trial

      Note
      • Each Alibaba Cloud account can apply for the free trial only once.

      • You can receive a seven-day free trial after your application is approved. The analytical data that is generated during the trial period is available only during the trial period. If you want to retain the analytical data, enable the bot management module before the trial period ends.

      On the Bot Traffic Analysis tab, click Apply for Trial. On the page that appears, enter the application information and click Submit.

      After you submit your trial application, Alibaba Cloud engineers will contact you based on the contact information that you submit. After the application is approved, the bot management module is automatically enabled for your WAF instance.

    • Enable the bot management module

      1. On the Bot Traffic Analysis, Scenario-specific Protection, or Basic Protection tab, click Purchase Now.

      2. On the buy page that appears, set the Bot Management - Web Application Protection or Bot Management - App Protection parameter to Enable and complete the payment.

        Note
        • After you enable bot management for web application protection, you can configure basic protection rules and anti-crawler rules for websites.

        • After you enable bot management for app protection, you can configure basic protection rules and anti-crawler rules for apps.

        • If you want to configure basic protection rules, anti-crawler rules for websites, and anti-crawler rules for apps, enable both bot management for web application protection and bot management for app protection.

After you enable the bot management module, you can configure scenario-specific protection policies on the Bot Traffic Analysis tab. To configure scenario-specific protection policies on the Bot Traffic Analysis tab, find the endpoint of the website or app that you want to protect and click Configure Protection in the Actions column. For more information, see Create an anti-crawler rule for websites and Create an anti-crawler rule for apps.

If you want to configure basic protection policies to defend against low-level crawlers, configure basic bot management rules on the Basic Protection tab. For more information, see Create a basic protection rule.

Create an anti-crawler rule for websites

If you want to use WAF to mitigate the security threats that are caused by bot traffic on web pages, HTML5 pages, or HTML5 apps, we recommend that you create an anti-crawler rule template for websites.

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and the region in which the WAF instance is deployed. The region can be Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Scenario-specific Protection > Bot Management.

  3. On the Scenario-specific Protection tab, click Create Template.

  4. In the Configure Scenarios step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Template Name

    Enter a name for the template.

    The name of the template must be 1 to 255 characters in length and can contain letters, digits, periods (.), underscores (_), and hyphens (-).

    Template Description

    Enter a description for the template.

    Service Type

    Select Websites. This way, WAF protects web pages, HTML5 pages, and HTML5 apps.

    Web SDK Integration

    Automatic Integration (recommended)

    WAF provides Web SDK for JavaScript to improve protection performance and prevent incompatibility issues.

    If you enable automatic integration, WAF automatically references the SDK in the HTML pages of the website that you want to protect. When automatic integration is enabled, the SDK collects information such as browser information, probe signatures, and malicious behaviors. Sensitive information is not collected. WAF detects and blocks malicious crawlers based on the collected information.

    If you use the domain name of another protected object to access the current protected object, you must select Use Intermediate Domain Name. Then, select the intermediate domain name from the drop-down list. For example, if you access Domain Name A from Domain Name B, you must select Use Intermediate Domain Name and select Domain Name B from the drop-down list.

    Important

    The automatic integration of Web SDK is not supported for Application Load Balancer (ALB) instances, Microservices Engine (MSE) instances, or custom domain names bound to web applications in Function Compute that are added to WAF.

    Manual Integration

    If automatic integration is not supported, you can use manual integration. You can click Obtain SDK to obtain the scripts and place the scripts above the other scripts. This ensures that the scripts are loaded first. For more information, see Deployment methods.

    For more information, see Integrate the Web SDK into web applications.

    Traffic Characteristics

    Add match conditions to identify traffic that is destined for the domain name that you want to protect. To add a match condition, you must configure the match field, logical operator, and match content. The match field is a header field of HTTP requests. You can add up to five conditions. The logical operator between the conditions is AND. For more information about match fields, see Match conditions.

  5. In the Configure Protection Rules step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Risk Identification

    Select Business Security and enter relevant information. For more information, see Risk identification.

    The feature helps block requests from abnormal mobile phone numbers based on Fraud Detection. You are charged based on rule hits.

    Legitimate Bot Management

    Select Spider Whitelist. Then, select search engines from the drop-down list.

    After you select Spider Whitelist and then select search engines from the drop-down list, requests that are sent from the crawler IP addresses of the search engines are sent to the origin server. The bot management module no longer checks these requests.

    Bot Characteristic Detection

    Script-based Bot Block (JavaScript Validation)

    If you select Script-based Bot Block (JavaScript Validation), WAF performs JavaScript validation on clients. To prevent simple script-based attacks, WAF blocks requests from non-browser tools that cannot run JavaScript.

    Advanced Bot Protection (Dynamic Token-based Authentication)

    If you select Advanced Bot Protection (Dynamic Token-based Authentication), WAF verifies the signature of each request. Requests that fail signature verification are blocked. Valid values:

    • Signature Verification Exception: This option is required. Requests that do not contain signatures or requests that contain invalid signatures are blocked.

    • Signature Timestamp Exception: Requests that contain abnormal signature timestamps are blocked.

    • WebDriver Attack: Requests are blocked when WebDriver attacks occur.

    Bot Behavior Detection

    Intelligent Protection

    If you select Intelligent Protection, you must select Monitor, Slider CAPTCHA, or Add Tag for the action that you want WAF to perform on detected bot requests. If you select Add Tag, you must enter a header name and header content.

    After you select Intelligent Protection, the intelligent protection engine analyzes access traffic and performs machine learning. Then, a blacklist or a protection rule is generated based on the analysis results and learned patterns.

    Custom Throttling

    You can configure custom throttling conditions to filter out crawler requests that are frequently initiated. This helps prevent HTTP flood attacks.

    • IP Address Throttling (Default)

      You can configure throttling conditions for IP addresses. If the number of requests that are sent from the same IP address within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Monitor, Slider CAPTCHA, or Block from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The logical operator between the conditions is OR.

    • Custom Session Throttling

      You can configure throttling conditions for sessions. You can configure the Session Type parameter to specify the session type. If the number of requests from the same session within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Monitor, Slider CAPTCHA, or Block from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The logical operator between the conditions is OR.

      The Session Type can be Custom Header, Custom Parameter, Custom Cookie, and Session.

    Bot Threat Intelligence

    Bot Threat Intelligence Library

    The library includes the IP addresses of attackers that have sent a large number of requests to crawl content from Alibaba Cloud users. If you select Bot Threat Intelligence Library, you can select Monitor, Slider CAPTCHA, or Add Tag for the action that you want WAF to perform on the requests. If you select Add Tag, you must enter a header name and header content.

    Data Center Blacklist

    If you select Data Center Blacklist, requests that are sent from the IP addresses in the selected IP address libraries are blocked. You can select Monitor, Slider CAPTCHA, Block, or Add Tag for the action that you want WAF to perform on the requests. If you select Add Tag, you must enter a header name and header content.

    Note

    If you use the source IP addresses of public clouds or data centers to access the website that you want to protect, you must add the IP addresses to the whitelist. For example, you must add the callback IP addresses of Alipay or WeChat and the IP addresses of monitoring applications to the whitelist.

    Fake Spider

    After you enable the feature, WAF blocks or adds tags to the user agents of all search engines in the Legitimate Bot Management section. If the IP addresses of the search engines are proved to be valid, WAF allows requests from the search engines.

  6. In the Configure Effective Scope step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Apply To

    Select the protected objects or protected object groups to which you want to apply the template in the Objects to Select section and click the 移入 icon to move the protected objects or protected object groups to the Selected Objects section.

    Effective Time and Canary Release

    You must specify validity periods and configure canary release settings for the protection rules. If you do not specify validity periods or configure canary release settings, canary release is disabled for the rules and the rules are permanently valid.

    1. Find the rule whose configurations you want to modify and click Edit in the Actions column.

    2. Configure canary release settings and specify validity periods.

      • Canary Release: You can turn on Canary Release to apply the rule to a specific proportion of objects.

        If you turn on Canary Release, you must configure the Dimension and Canary Release Ratio parameters. Valid values of the Dimension parameter: IP Address, Custom Header, Custom Header, Custom Cookie, and Session.

      • Effective Mode

        • Permanently Effective (default): If you select Permanently Effective, the protection rule is permanently in effect.

        • Fixed Schedule: You can specify a time zone and a period of time during which you want the template to be in effect.

        • Recurring Schedule: You can specify a time zone, days of the week, and a period of time in a day. The template is in effect during the same period of time in a day on the same days of the week.

    You can select multiple rules to specify validity periods and configure canary release settings for the rules at the same time.

  7. In the Verify Protection Effect step, test the anti-crawler rule.

    Before you publish an anti-crawler rule, we recommend that you verify the protection effect of the rule to prevent false positives that are caused by improper rule configurations or compatibility issues. If the configurations are correct, click Skip to skip this step.

    Test steps:

    1. Step 1: Enter a public IP address.

      Enter the public IP address of your test device. The test device can be a computer or mobile phone. The test takes effect only for the public IP address. The test does not affect your business.

      Important

      Do not enter the IP address that you obtain by running the ipconfig command. This command returns a private IP address. If you are unsure of the public IP address of your test device, you can use an online IP lookup tool to query the public IP address.

    2. Step 2: Select an action.

      Test the effectiveness of the action that you specified in the Configure Protection Rules step. WAF generates a test rule only for the specified IP address. The action can be JavaScript Validation, Dynamic Token-based Authentication, Slider CAPTCHA Verification, or Block Verification.

      After you click Test for an action, WAF immediately generates a test rule and sends a test request to the test device. In the dialog box that appears, WAF provides the test procedure, expected result, and demonstration. We recommend that you carefully read the information in the dialog box.

      After the test is complete, click I Have Completed the Test to go to the next step. If the test result shows exceptions, click Go Back to optimize the anti-crawler rule based on the instructions that are described in the "FAQ" section. Then, perform the test again.

By default, the new template is enabled. On the Scenario-specific Protection tab, you can perform the following operations in the template card:

  • Click a template card to view the rule information about the template.

  • Copy, modify, or delete the template.

  • Turn on or turn off the switch to enable or disable the template.

  • View the action that is specified in the template and the protected objects or groups that are associated with the template.

Create an anti-crawler rule for apps

You can configure anti-crawler rules for native iOS or Android apps to protect your services against crawlers. HTML5 apps are not native iOS or Android apps.

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and the region in which the WAF instance is deployed. The region can be Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Scenario-specific Protection > Bot Management.

  3. On the Scenario-specific Protection tab, click Create Template.

  4. In the Configure Scenarios step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Template Name

    Enter a name for the template.

    The name of the template must be 1 to 255 characters in length and can contain letters, digits, periods (.), underscores (_), and hyphens (-).

    Template Description

    Enter a description for the template.

    Service Type

    Select App to configure anti-crawler rules for native iOS and Android apps. HTML5 apps are not native iOS or Android apps.

    App SDK Integration

    WAF provides the Anti-Bot SDK to enhance protection capabilities for native Android and iOS apps. After the Anti-Bot SDK is integrated into apps, the Anti-Bot SDK collects the characteristics of clients and generates security signatures in requests. WAF identifies and blocks requests that are identified as unsafe based on the signatures.

    You can perform the following steps to integrate the Anti-Bot SDK.

    1. Obtain the SDK for iOS apps. To obtain the SDK for iOS apps, submit a ticket.

    2. Click Obtain and Copy AppKey to send SDK initialization requests.

    3. Integrate the Anti-Bot SDK into iOS apps. For more information, see Integrate the Anti-Bot SDK into iOS apps.

    Traffic Characteristics

    Add match conditions to identify traffic that is destined for the domain name that you want to protect. To add a match condition, you must configure the match field, logical operator, and match content. The match field is a header field of HTTP requests. You can add up to five conditions. The logical operator between the conditions is AND. For more information about match fields, see Match conditions.

  5. In the Configure Protection Rules step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Risk Identification

    Select Business Security and enter relevant information. For more information, see Risk identification.

    The feature helps block requests from abnormal mobile phone numbers based on Fraud Detection. You are charged based on rule hits.

    Bot Characteristic Detection

    • Detection rules

      Invalid App Signature. By default, this is selected.

      By default, this option is selected and cannot be deselected. After the Anti-Bot SDK is integrated, WAF can detect requests that contain invalid signatures or do not contain signatures.

      Custom Signature Field. By default, this is not selected.

      If you select Custom Signature Field, you must select Cookie, Parameter, or Header from the Field Name drop-down list and enter the custom content in the Value field. If the body of the request to be signed is empty, contains special characters, or exceeds the length limit, you can hash the request and enter the returned string in the Value field.

      Abnormal Device Behavior

      After you enable this feature, WAF detects and controls the requests from the devices that have abnormal behaviors. The behaviors include Expired Signature, Using Simulator, Using Proxy, Rooted Device, Debugging Mode, Hooking, Multiboxing, Simulated Execution, and Script Tools.

    • Protection action

      You can select Monitor, Block, or Strict Slider CAPTCHA Verification for the action that you want WAF to perform on the requests that match the Bot Characteristic Detection rule.

    • Advanced protection

      Click Advanced Protection and configure the following parameters:

      Secondary Packaging Detection

      • Rule settings

        Requests that are sent from apps whose package names or package signatures are not in the whitelist are considered repackaging requests. Specify valid app packages.

        • Valid Package Name: Enter a valid application package name. Example: example.aliyundoc.com.

        • Signature specify the app package signature that needs to be verified. If the signature needs to be verified, submit a ticket. If the package signature does not need to be verified, leave this parameter empty. If the parameter is left empty, WAF verifies only the package name.

        • Important

          The value of Signature is not the certificate-based signature of the app.

        You can add up to five valid iOS or Android app packages. The package names must be unique. The logical operator between the conditions is OR.

      • Protection action

        You can select Monitor, Block, Slider CAPTCHA, or Strict Slider CAPTCHA Verification for the action that you want WAF to perform on the requests that match the repackaging detection rule.

      Custom Rule

      If the default settings cannot meet your protection requirements, you can configure custom rules. To configure custom rules, select Custom Rule, click Create Rule, and configure the following parameters.

      • Match Condition: You can add up to five match conditions. The logical operator between the conditions is AND.

        Click to view the supported match fields.

        eeid_is_root: specifies whether the device has root permissions.

        eeid_is_proxy: specifies whether the device is a proxy.

        eeid_is_simulator: specifies whether the device is a simulator.

        eeid_is_debugged: specifies whether the debugging mode is used.

        eeid_is_hook: specifies whether Hooking techniques are used.

        eeid_is_virtual: specifies whether multiple app processes are running on the device at the same time.

        eeid_is_new: specifies whether the device is a new device.

        eeid_is_wiped: specifies whether the device is suspected of brushing.

        eeid_short_uptime: specifies whether the startup time is excessively short.

        eeid_abnormal_time: specifies whether the system time is abnormal.

        eeid_running_frame_xposed: specifies whether Xposed is used.

        eeid_running_frame_frida: specifies whether Frida is used.

        eeid_running_frame_cydia: specifies whether Cydia is used.

        eeid_running_frame_fishhook: specifies whether fishhook is used.

        eeid_running_frame_va: specifies whether VA Framework is used.

        eeid_running_frame_magisk: specifies whether Magisk is used.

        eeid_running_frame_edxposed: specifies whether EdXposed is used.

        eeid_umid: specifies whether UMID is used.

        appname: specifies the application name.

        packagename: specifies the package name.

        appversion: specifies the version of the app.

        version: specifies the version of the SDK.

        brand: specifies the mobile phone brand.

        model: specifies the mobile phone model.

        product: specifies the product code.

        manufacture: specifies the mobile phone manufacturer.

        hardware: specifies the hardware name.

      • Action: You can select Monitor, Block, Slider CAPTCHA, Strict Slider CAPTCHA Verification, or Add Tag for the action that you want WAF to perform on the requests that match the rule. If you select Add Tag, you must enter a header name and header content.

      You can add up to 10 custom rules. The logical operator between the rules is OR.

    Bot Behavior Detection

    If you select Intelligent Protection, you must select Monitor, Slider CAPTCHA, Strict Slider CAPTCHA Verification, or Add Tag for the action that you want WAF to perform on detected bot requests. If you select Add Tag, you must enter a header name and header content.

    After you select Intelligent Protection, the intelligent protection engine analyzes access traffic and performs machine learning. Then, a blacklist or a protection rule is generated based on the analysis results and learned patterns.

    Throttling

    You can configure custom throttling conditions to filter out crawler requests that are frequently initiated. This helps prevent HTTP flood attacks.

    • IP Address Throttling (Default)

      You can configure throttling conditions for IP addresses. If the number of requests that are sent from the same IP address within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Block, Monitor, Slider CAPTCHA, or Strict Slider CAPTCHA Verification from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The logical operator between the conditions is OR.

    • Device Throttling

      You can configure throttling conditions for devices. If the number of requests that are sent from the same device within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Block, Monitor, Slider CAPTCHA, or Strict Slider CAPTCHA Verification from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The logical operator between the conditions is OR.

    • Custom Session Throttling

      You can configure throttling conditions for sessions. You can configure the Session Type parameter to specify the session type. If the number of requests that are sent from the same session within the value of the Statistical Interval (Seconds) parameter exceeds the value of the Threshold (Times) parameter, WAF performs the specified action on subsequent requests. You can select Block, Monitor, Slider CAPTCHA, or Strict Slider CAPTCHA Verification from the Action drop-down list to specify the action that you want WAF to perform. You can also configure the Throttling Interval (Seconds) parameter to specify the period of time during which the specified action is performed. You can add up to three conditions. The logical operator between the conditions is OR.

      The Session Type can be Custom Header, Custom Parameter, Custom Cookie, and Session.

    Bot Threat Intelligence

    Bot Threat Intelligence Library

    The library includes the IP addresses of attackers that have sent multiple requests to crawl content from Alibaba Cloud users. If you select Bot Threat Intelligence Library, you can select Monitor, Slider CAPTCHA, Strict Slider CAPTCHA Verification, or Add Tag for the action that you want WAF to perform on the requests. If you select Add Tag, you must enter a header name and header content.

    Data Center Blacklist

    If you select Data Center Blacklist, requests that are sent from the IP addresses in the selected IP address libraries of data centers are blocked. You can select Monitor, Slider CAPTCHA, Block, Strict Slider CAPTCHA Verification, or Add Tag for the action that you want WAF to perform on the requests. If you select Add Tag, you must enter a header name and header content.

    Note

    If you use the source IP addresses of public clouds or data centers to access the website that you want to protect, you must add the IP addresses to the whitelist. For example, you must add the callback IP addresses of Alipay or WeChat and the IP addresses of monitoring applications to the whitelist.

  6. In the Configure Effective Scope step, configure the parameters and click Next. The following table describes the parameters.

    Parameter

    Description

    Apply To

    Select the protected objects or protected object groups to which you want to apply the template in the Objects to Select section and click the 移入 icon to move the protected objects or protected object groups to the Selected Objects section.

    Effective Time and Canary Release

    You must specify validity periods and configure canary release settings for the protection rules. If you do not specify validity periods or configure canary release settings, canary release is disabled for the rules and the rules are permanently valid.

    1. Find the rule whose configurations you want to modify and click Edit in the Actions column.

    2. Configure canary release settings and specify validity periods.

      • Canary Release: You can turn on Canary Release to apply the rule to a specific proportion of objects.

        If you turn on Canary Release, you must configure the Dimension and Canary Release Ratio parameters. Valid values of the Dimension parameter: IP Address, Custom Header, Custom Header, Custom Cookie, and Session.

      • Effective Mode

        • Permanently Effective (default): If you select Permanently Effective, the protection rule is permanently in effect.

        • Fixed Schedule: You can specify a time zone and a period of time during which you want the template to be in effect.

        • Recurring Schedule: You can specify a time zone, days of the week, and a period of time in a day. The template is in effect during the same period of time in a day on the same days of the week.

    You can select multiple rules to specify validity periods and configure canary release settings for the rules at the same time.

  7. In the Verify Protection Effect step, test the anti-crawler rule.

    Before you publish an anti-crawler rule, we recommend that you verify the protection effect of the rule to prevent false positives that are caused by improper rule configurations or compatibility issues. If the configurations are correct, click Skip to skip this step.

    Test steps:

    1. Step 1: Enter a public IP address.

      Enter the public IP address of your test device. The test device can be a computer or mobile phone. The test takes effect only for the public IP address. The test does not affect your business.

      Important

      Do not enter the IP address that you obtain by running the ipconfig command. This command returns a private IP address. If you are unsure of the public IP address of your test device, you can use an online IP lookup tool to query the public IP address.

    2. Step 2: Select an action.

      A test rule is generated based on the configurations in the Configure Protection Rules step. You can select Block Verification or SDK Signature Verification for the action that you want WAF to perform on the requests that match the test rule. The test rule takes effect only for the test IP address.

      After you click Test for an action, WAF immediately generates a test rule and sends a test request to the test device. In the dialog box that appears, WAF provides the test procedure, expected result, and demonstration. We recommend that you carefully read the information in the dialog box.

      After the test is complete, click I Have Completed the Test the Test to go to the next step. If the test result shows exceptions, click Go Back to optimize the anti-crawler rule based on the instructions that are described in the "FAQ" section in this topic. Then, perform the test again.

By default, the new template is enabled. On the Scenario-specific Protection tab, you can perform the following operations in the template card:

  • Click a template card to view the rule information about the template.

  • Copy, modify, or delete the template.

  • Turn on or turn off the switch to enable or disable the template.

  • View the action that is specified in the template and the protected objects or groups that are associated with the template.

Create a basic protection rule

You can configure basic protection rules to defend against medium- and low-level crawlers for your services. The bot management module does not provide a default basic protection rule template. Before you can enable the basic protection feature provided by the bot management module, you must create a basic protection rule template.

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and the region in which the WAF instance is deployed. The region can be Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Protection Configuration > Scenario-specific Protection > Bot Management.

  3. On the Basic Protection tab, click Create Template.

  4. In the Create Template - Bot Management panel, configure the parameters and click OK. The following table describes the parameters.

    Parameter

    Description

    Template Name

    Enter a name for the template.

    The name of the template must be 1 to 255 characters in length and can contain letters, digits, periods (.), underscores (_), and hyphens (-).

    Template Description

    Enter a description for the template.

    Action

    Specify an action that you want WAF to perform on the requests that match the rule. Valid values: Block and Monitor.

    Advanced Settings

    • Canary Release: You can turn on Canary Release to apply the rule to a specific proportion of objects.

      If you turn on Canary Release, you must configure the Dimension and Canary Release Ratio parameters. Valid values of the Dimension parameter: IP Address, Custom Header, Custom Header, Custom Cookie, and Session.

    • Effective Mode

      • Permanently Effective (default): If you select Permanently Effective, the protection rule is permanently in effect.

      • Fixed Schedule: You can specify a time zone and a period of time during which you want the template to be in effect.

      • Recurring Schedule: You can specify a time zone, days of the week, and a period of time in a day. The template is in effect during the same period of time in a day on the same days of the week.

    Apply To

    Select the protected objects and protected object groups to which you want to apply the template.

    You can apply only one template of a protection module to a protected object or a protected object group. For information about how to associate protected objects and protected object groups with the template, see Protected objects and protected object groups.

By default, the new template is enabled. On the Basic Protection tab, you can perform the following operations in the template card:

  • View the IDs of the rules that are included in the template.

    Note

    A basic protection rule template includes two whitelist rules and one access control or HTTP flood protection rule. You can view the protection performance of the rules on the Security Reports page by rule ID. For more information, see Security reports.

  • Copy, modify, or delete the template.

  • Turn on or turn off the switch to enable or disable the template.

  • View the action that is specified in the template and the protected objects or goups that are associated with the template.

FAQ

If an issue occurs in the Verify Protection Effect step, refer to the following table to resolve the issue.

Error

Cause

Solution

No valid test requests are detected. See WAF documentation or contact us to identify the possible causes.

The test request failed to be sent or is not sent to WAF.

Make sure that the test request is sent to the IP address that maps the CNAME provided by WAF.

The header fields in the test request do not match the header fields that you configured for the Traffic Characteristics parameter in the anti-crawler rule.

Modify the Traffic Characteristics parameter in the anti-crawler rule.

The source IP address of the test request is different from the public IP address that you specified in the anti-crawler rule.

Make sure that you use the correct public IP address. We recommend that you use Alibaba Network Diagnose Tool to obtain your public IP address.

The test requests failed the verification. See WAF documentation or contact us to identify the possible causes.

No real user access is simulated. For example, the debugging mode or automation tools are used.

Simulate a real user to access your website or app during the test.

An incorrect service type is selected. For example, Websites is selected when you configure an anti-crawler rule for apps.

Modify the value of the Service Type parameter.

An intermediate domain name is used, but an incorrect intermediate domain name is selected in the anti-crawler rule.

Select Use Intermediate Domain Name. Then, select the correct intermediate domain name from the drop-down list.

Compatibility issues occur in the frontend.

Submit a ticket to contact us.

No verification is triggered. See WAF documentation or contact us to identify the possible causes.

No test rules are generated.

Perform the test several times until a test rule is generated.

No valid test requests are detected or blocked. See WAF documentation or contact us to identify the possible causes.

The test request failed to be sent or is not sent to WAF.

Make sure that the test request is sent to the IP address that maps the CNAME provided by WAF.

The header fields in the test request do not match the header fields that you configured for the Traffic Characteristics parameter in the anti-crawler rule.

Modify the Traffic Characteristics parameter in the anti-crawler rule.

The source IP address of the test request is different from the public IP address that you specified in the anti-crawler rule.

Make sure that you use the correct public IP address. We recommend that you use Alibaba Network Diagnose Tool to obtain your public IP address.

What to do next

On the Security Reports page, you can query the protection details of the protection rules that you configured. For more information, see Security reports.