Configure log collection policies

Supported Alibaba Cloud services

Configure a log collection policy

1. Log on to the Log Service console.
2. In the Log Application section, click Log Audit Service.
3. Choose Access to Cloud Products > Global Configurations. In the upper-right corner of the page that appears, click Modify.
4. Find the Alibaba Cloud service for which you want to configure a log collection policy and click Collection Policy.
5. Configure a log collection policy.
   Log Service provides the basic edit mode and the advanced edit mode for you to configure a log collection policy. The configuration in basic edit mode is simple. If the basic edit mode does not meet your business requirements, you can enable the advanced edit mode. In this mode, you can configure the log collection policy that is required in complex scenarios in a flexible manner.

   Note

   • You can configure multiple policies based on your business requirements.
   • In advanced edit mode, you can edit policy statements. After you edit a policy statement, you cannot return to the basic edit mode.
   • To return to the basic edit mode, you must delete all policy statements and save the changes. Then, click Collection Policy again. The basic edit mode is in effect.

   • Basic edit mode
     1. In the Add Policy section, configure the following parameters and click Add Policy.

        Parameter	Description
        Action	The action that is performed when Log Service collects logs based on the log collection policy that you configure. For more information, see Policy syntax.
        Properties	The property of the log source. Available properties vary based on the log source that you use. For more information, see Supported Alibaba Cloud services.
        Operator	The match mode that corresponds to an operator. If you select Exact Match, the operator is ==. For more information, see Policy syntax.
        Property value	The value of the property. You can specify multiple values for a property.

     2. In the Added Policies section, confirm the log collection policy that you configure.
        You can modify the policy and change the order of the policy.

        • To modify the policy, click Edit on the right of the policy.
        • To change the order of the policy, click the upward or downward arrow on the right of the policy.

        
        Note The accept "*" policy evaluates to true for all log sources. This is the default log collection policy. You cannot edit or delete this policy.
     3. After you confirm the policy, click OK.
   • Advanced edit mode
     1. Turn on Advanced Edit Mode.
     2. In the Rule field, configure a log collection policy and click OK.

        For more information about the policy syntax, see Policy syntax.

        
6. On the Global Configurations page, click OK.

Policy syntax

• Actions
  • Keep: If the log source matches a policy, Log Service attempts to match the log source against the next policy and determines whether to collect logs based on subsequent policies. If the log source does not match the policy, Log Service does not collect logs and no longer attempts to match the log source against subsequent policies.
  • Drop: If the log source matches a policy, Log Service does not collect logs and no longer attempts to match the log source against subsequent policies. If the log source does not match the policy, Log Service attempts to match the log source against the next policy and determines whether to collect logs based on subsequent policies.
  • Accept: If the log source matches a policy, Log Service collects logs and no longer attempts to match the log source against subsequent policies. If the log source does not match the policy, Log Service attempts to match the log source against the next policy and determines whether to collect logs based on subsequent policies.
• Match modes

  Match mode	Description
  Exact match	The exact match of strings. Operator: ==. Example: keep instance.db_type == "mysql". This policy evaluates to true for an ApsaraDB RDS for MySQL instance.
  Wildcard match	The match of data based on wildcard characters. The wildcard characters include asterisks (*) and question marks (?). An asterisk (*) specifies zero or multiple characters. A question mark (?) specifies one character. Operator: ==. Examples: keep instance.name == "backend*". This policy evaluates to true for an instance whose name starts with backend. keep instance.name == "active?". This policy evaluates to true for an instance whose name starts with active and a random character.
  Regex match	The match of data based on regular expressions. Operator: ~=. keep instance.name ~= "^\d+$". This policy evaluates to true for an instance whose name contains only digits. Note By default, Log Service performs partial match. To enable exact match, you must prefix a regular expression with a caret (^) and suffix the regular expression with a dollar sign ($).
  Numeric value comparison	The comparison of numeric values. Operators: Operators for direct comparison: greater-than (>), greater-than-or-equal to (>=), equal-to (=), less-than-or-equal-to (<=), and less-than (<). Operators for comparing numeric values within a closed interval. Example: : [*, 100]. An asterisk (*) can be used to specify an infinite interval. Examples: keep tag.level >= 2. This policy evaluates to true for an instance whose value of the tag.level property is greater than or equal to 2. keep tag.level : [*, 10]. This policy evaluates to true for an instance whose value of the tag.level property is less than or equal to 10. keep tag.level : [1, 10]. This policy evaluates to true for an instance whose value of the tag.level property is within the closed interval [1, 10].
  Logical judgment	Keywords: and, AND, and &&: The keywords are not case-sensitive. or and OR: The keywords are not case-sensitive. not, NOT, and exclamation point (!): The keywords are not case-sensitive. Examples: keep (tag.level > 10) and (region == "cn-shanghai"). This policy evaluates to true for an instance whose value of the tag.level property is greater than 10 and that resides in the China (Shanghai) region. keep (tag.level > 10) or (region == "cn-shanghai"). This policy evaluates to true for an instance whose value of the tag.level property is greater than 10 or that resides in the China (Shanghai) region. keep not region == "cn-shanghai". This policy evaluates to true for an instance that does not reside in the China (Shanghai) region.
  Global match	If no property is specified in a log collection policy, the system matches log sources against all available properties for the policy. Examples: keep "abc". This policy evaluates to true for logs that contain the abc string. accept "*". This policy evaluates to true for all log sources. Note If you use global match, you must enclose specified characters in double quotation marks (""). Global match is available only in advanced edit mode.

• Character escape

  If a log collection policy contains special characters such as asterisks (*) and backslashes (\), the special characters must be escaped. Example: keep instance.name == "abc\*". This policy evaluates to true for an instance whose name is abc*.

Common scenarios

• Collect the logs of instances that reside in a specified region
  In the following example, only the logs of instances that reside in the China region are collected based on the configured collection policies.

  # only scan cn region
  keep region == "cn-*"
  
  # accept by default
  accept "*"

• Collect the logs of instances that have specified tags
  In the following example, only the logs of instances whose value of type tag is production are collected based on the configured collection policies. The value production is not case-sensitive.

  # only scan "production" instances
  keep tag.type ~= "(?i)^production$"
  
  # accept by default
  accept "*"

• Complex scenarios
  If the level: high tag is used in log collection policies, the logs of ApsaraDB RDS for MySQL instances, ApsaraDB RDS for SQL Server instances, and ApsaraDB RDS for PostgreSQL instances are collected. If the level: high tag is not used, only the logs of ApsaraDB RDS for MySQL instances are collected. The following code shows the log collection policies that are involved:

  # accept all high level instances
  accept tag.level == "high"
  
  # only scan mysql
  keep instance.db_type == "mysql"
  
  # accept by default
  accept "*"