The Log Audit Service application allows you to collect logs from Alibaba Cloud services across multiple accounts and store the logs in a centralized manner. If the log audit feature is enabled for an Alibaba Cloud service, Log Service collects all logs that meet specified conditions from the service by default. You can configure log collection policies to specify the accounts, regions, and instances from which logs are collected. This way, you can collect logs at a fine-grained level. This topic describes how to configure log collection policies.
Supported Alibaba Cloud services
Alibaba Cloud service | Log source | Property | Description |
---|---|---|---|
ApsaraDB RDS | ApsaraDB RDS instance | account.id | The ID of the Alibaba Cloud account to which the ApsaraDB RDS instance belongs. |
region | The ID of the region where the ApsaraDB RDS instance resides. Example: cn-shanghai. | ||
instance.id | The ID of the ApsaraDB RDS instance. | ||
instance.name | The name of the ApsaraDB RDS instance. | ||
instance.db_type | The type of the databases that are created on the ApsaraDB RDS instance. Valid values: mysql, pgsql, and mssql. | ||
instance.db_version | The version of the database engine. Example: 8.0. | ||
tag.* | The custom tag.
You can replace the asterisk (*) in the tag.* property with a custom tag name. |
||
PolarDB | PolarDB cluster | account.id | The ID of the Alibaba Cloud account to which the PolarDB cluster belongs. |
region | The ID of the region where the PolarDB cluster resides. Example: cn-shanghai. | ||
cluster.id | The ID of the PolarDB cluster. | ||
cluster.name | The name of the PolarDB cluster. | ||
cluster.db_type | The database type that is supported by the PolarDB cluster. Valid values: MySQL. | ||
cluster.db_version | The version of the database engine. Valid values: 5.6, 5.7, and 8.0. | ||
tag.* | The custom tag.
You can replace the asterisk (*) in the tag.* property with a custom tag name. |
||
PolarDB-X 1.0 | PolarDB-X 1.0 instance | account.id | The ID of the Alibaba Cloud account to which the PolarDB-X 1.0 instance belongs. |
region | The ID of the region where the PolarDB-X 1.0 instance resides. Example: cn-shanghai. | ||
instance.id | The ID of the PolarDB-X 1.0 instance. | ||
instance.name | The name of the PolarDB-X 1.0 instance. | ||
SLB | SLB instance | account.id | The ID of the Alibaba Cloud account to which the SLB instance belongs. |
region | The ID of the region where the SLB instance resides. Example: cn-shanghai. | ||
instance.id | The ID of the SLB instance. | ||
instance.name | The name of the SLB instance. | ||
instance.network_type | The network type of the SLB instance. Valid values: vpc and classic. | ||
instance.vpc_id | The ID of the VPC where the SLB instance resides. | ||
instance.address_type | The address type of the SLB instance. Valid values: intranet and internet. | ||
tag.* | The custom tag.
You can replace the asterisk (*) in the tag.* property with a custom tag name. |
||
ALB | ALB instance | account.id | The ID of the Alibaba Cloud account to which the ALB instance belongs. |
region | The ID of the region where the ALB instance resides. Example: cn-shanghai. | ||
instance.id | The ID of the ALB instance. | ||
instance.name | The name of the ALB instance. | ||
instance.vpc_id | The ID of the VPC where the ALB instance resides. | ||
instance.address_type | The address type of the ALB instance. Valid values: Intranet and Internet. | ||
tag.* | The custom tag.
You can replace the asterisk (*) in the tag.* property with a custom tag name. |
||
VPC | VPC | account.id | The ID of the Alibaba Cloud account to which the VPC belongs. |
region | The ID of the region where the VPC resides. | ||
instance.id | The ID of the VPC. | ||
instance.name | The name of the VPC. | ||
tag * | The custom tag.
You can replace the asterisk (*) in the tag.* property with a custom tag name. |
||
ACK (Kubernetes audit log) | Kubernetes cluster | region | The ID of the region where the Kubernetes cluster resides. Example: cn-shanghai. |
cluster.id | The ID of the Kubernetes cluster. | ||
cluster.name | The name of the Kubernetes cluster. | ||
cluster.type | The type of the Kubernetes cluster. Valid values: Kubernetes, ManagedKubernetes, and ASK. | ||
cluster.network_mode | The network type of the Kubernetes cluster. Valid values: vpc and classic. | ||
tag.* | The custom tag.
You can replace the asterisk (*) in the tag.* property with a custom tag name. |
||
ACK (Kubernetes event center) | Kubernetes cluster | region | The ID of the region where the Kubernetes cluster resides. Example: cn-shanghai. |
cluster.id | The ID of the Kubernetes cluster. | ||
cluster.name | The name of the Kubernetes cluster. | ||
cluster.type | The type of the Kubernetes cluster. Valid values: Kubernetes, ManagedKubernetes, and ASK. | ||
cluster.network_mode | The network type of the Kubernetes cluster. Valid values: vpc and classic. | ||
tag.* | The custom tag.
You can replace the asterisk (*) in the tag.* property with a custom tag name. |
||
ACK (Ingress access log) | Kubernetes cluster | region | The ID of the region where the Kubernetes cluster resides. Example: cn-shanghai. |
cluster.id | The ID of the Kubernetes cluster. | ||
cluster.name | The name of the Kubernetes cluster. | ||
cluster.type | The type of the Kubernetes cluster. Valid values: Kubernetes, ManagedKubernetes, and ASK. | ||
cluster.network_mode | The network type of the Kubernetes cluster. Valid values: vpc and classic. | ||
tag.* | The custom tag.
You can replace the asterisk (*) in the tag.* property with a custom tag name. |
||
log.* | The content of the log. |
Configure a log collection policy
Policy syntax
- Actions
- Keep: If the log source matches a policy, Log Service attempts to match the log source against the next policy and determines whether to collect logs based on subsequent policies. If the log source does not match the policy, Log Service does not collect logs and no longer attempts to match the log source against subsequent policies.
- Drop: If the log source matches a policy, Log Service does not collect logs and no longer attempts to match the log source against subsequent policies. If the log source does not match the policy, Log Service attempts to match the log source against the next policy and determines whether to collect logs based on subsequent policies.
- Accept: If the log source matches a policy, Log Service collects logs and no longer attempts to match the log source against subsequent policies. If the log source does not match the policy, Log Service attempts to match the log source against the next policy and determines whether to collect logs based on subsequent policies.
- Match modes
Match mode Description Exact match The exact match of strings. - Operator: ==.
- Example: keep instance.db_type == "mysql". This policy evaluates to true for an ApsaraDB RDS for MySQL instance.
Wildcard match The match of data based on wildcard characters. The wildcard characters include asterisks (*) and question marks (?). An asterisk (*) specifies zero or multiple characters. A question mark (?) specifies one character. - Operator: ==.
- Examples:
- keep instance.name == "backend*". This policy evaluates to true for an instance whose name starts with backend.
- keep instance.name == "active?". This policy evaluates to true for an instance whose name starts with active and a random character.
Regex match The match of data based on regular expressions. - Operator: ~=.
- keep instance.name ~= "^\d+$". This policy evaluates to true for an instance whose name contains only digits.
Note By default, Log Service performs partial match. To enable exact match, you must prefix a regular expression with a caret (^) and suffix the regular expression with a dollar sign ($).Numeric value comparison The comparison of numeric values. - Operators:
- Operators for direct comparison: greater-than (>), greater-than-or-equal to (>=), equal-to (=), less-than-or-equal-to (<=), and less-than (<).
- Operators for comparing numeric values within a closed interval. Example: : [*, 100]. An asterisk (*) can be used to specify an infinite interval.
- Examples:
- keep tag.level >= 2. This policy evaluates to true for an instance whose value of the tag.level property is greater than or equal to 2.
- keep tag.level : [*, 10]. This policy evaluates to true for an instance whose value of the tag.level property is less than or equal to 10.
- keep tag.level : [1, 10]. This policy evaluates to true for an instance whose value of the tag.level property is within the closed interval [1, 10].
Logical judgment - Keywords:
- and, AND, and &&: The keywords are not case-sensitive.
- or and OR: The keywords are not case-sensitive.
- not, NOT, and exclamation point (!): The keywords are not case-sensitive.
- Examples:
- keep (tag.level > 10) and (region == "cn-shanghai"). This policy evaluates to true for an instance whose value of the tag.level property is greater than 10 and that resides in the China (Shanghai) region.
- keep (tag.level > 10) or (region == "cn-shanghai"). This policy evaluates to true for an instance whose value of the tag.level property is greater than 10 or that resides in the China (Shanghai) region.
- keep not region == "cn-shanghai". This policy evaluates to true for an instance that does not reside in the China (Shanghai) region.
Global match If no property is specified in a log collection policy, the system matches log sources against all available properties for the policy. Examples: - keep "abc". This policy evaluates to true for logs that contain the abc string.
- accept "*". This policy evaluates to true for all log sources.
Note- If you use global match, you must enclose specified characters in double quotation marks ("").
- Global match is available only in advanced edit mode.
- Character escape
If a log collection policy contains special characters such as asterisks (*) and backslashes (\), the special characters must be escaped. Example: keep instance.name == "abc\*". This policy evaluates to true for an instance whose name is abc*.
Common scenarios
- Collect the logs of instances that reside in a specified region
In the following example, only the logs of instances that reside in the China region are collected based on the configured collection policies.
# only scan cn region keep region == "cn-*" # accept by default accept "*"
- Collect the logs of instances that have specified tags
In the following example, only the logs of instances whose value of type tag is production are collected based on the configured collection policies. The value production is not case-sensitive.
# only scan "production" instances keep tag.type ~= "(?i)^production$" # accept by default accept "*"
- Complex scenarios
If the level: high tag is used in log collection policies, the logs of ApsaraDB RDS for MySQL instances, ApsaraDB RDS for SQL Server instances, and ApsaraDB RDS for PostgreSQL instances are collected. If the level: high tag is not used, only the logs of ApsaraDB RDS for MySQL instances are collected. The following code shows the log collection policies that are involved:
# accept all high level instances accept tag.level == "high" # only scan mysql keep instance.db_type == "mysql" # accept by default accept "*"