Parameter | Description |
Logtail Deployment Mode | Select the deployment mode of Logtail. In this example, select Daemonset. |
File Path Type | Select the file path type of the logs that you want to collect. Valid values: Path in Container and Host Path. In this example, select Path in Container. |
File Path | Specify the directory and name of log files. Configure this parameter based on the location of the logs that you want to collect. If the container runs on a Linux host, the log path must start with a forward slash (/). Example: /apsara/nuwa/**/app.Log . If the container runs on a Windows host, the log path must start with a drive letter. Example: C:\Program Files\Intel\**\*.Log .
You can specify an exact directory and an exact name. You can also use wildcard characters to specify the directory and name. For more information, see Wildcard matching. When you configure this parameter, you can use only asterisks (*) or question marks (?) as wildcard characters. Simple Log Service scans all levels of the specified directory for the log files that match specified conditions. Examples: If you specify /apsara/nuwa/**/*.log , Simple Log Service collects logs from the log files whose names are suffixed by .log in the /apsara/nuwa directory and the recursive subdirectories of the directory. If you specify /var/logs/app_*/**/*.log , Simple Log Service collects logs from the log files that meet the following conditions: The file name is suffixed by .log . The file is stored in a subdirectory under the /var/logs directory or in a recursive subdirectory of the subdirectory. The name of the subdirectory matches the app_* pattern. If you specify /var/log/nginx/**/access* , Simple Log Service collects logs from the log files whose names start with access in the /var/log/nginx directory and the recursive subdirectories of the directory.
|
Maximum Directory Monitoring Depth | Specify the maximum number of levels of subdirectories that you want to monitor. The subdirectories are in the log file directory that you specify. This parameter specifies the levels of subdirectories that can be matched for the wildcard characters ** included in the value of File Path. A value of 0 specifies that only the log file directory that you specify is monitored. |
Multi-line Mode | If you want to collect multi-line logs, turn on Multi-line Mode. |
Regex to Match First Line | Configure a regular expression to match the beginning of the first line of a log. Logtail uses the regular expression to match the beginning of the first line of a log and considers the content that does not match the regular expression as part of the log. Simple Log Service can automatically generate a regular expression or use the regular expression that you manually specify. Automatic generation Click Auto Generate. Simple Log Service automatically generates a regular expression based on your sample log to match the beginning of the first line of a log. Manual configuration Click Manual and manually specify a regular expression to match the beginning of the first line of a log. Then, click Validate to check whether the regular expression is valid. For more information, see How do I test a regular expression?
|
Enable Container Metadata Preview | If you turn on Enable Container Metadata Preview, you can view the container metadata after you create the Logtail configuration, including the matched container information and full container information. |
Container Filtering | Specify conditions to filter containers. For versions earlier than Logtail V1.0.34, you can filter containers only by using environment variables and container labels. For Logtail V1.0.34 and later, we recommend that you filter containers by using different levels of Kubernetes information, such as pod names, Kubernetes namespaces, container names, and labels.
A namespace of a Kubernetes cluster and the name of a container in a Kubernetes cluster can be mapped to container labels. The value of the Label Name parameter for a namespace is io.kubernetes.pod.namespace . The value of the Label Name parameter for a container name is io.kubernetes.container.name . We recommend that you use the two container labels to filter containers. If the container labels do not meet your business requirements, you can use the environment variable whitelist or the environment variable blacklist to filter containers. For example, the namespace of a pod is backend-prod, and the name of a container in the pod is worker-server. If you want the logs of the worker-server container to be collected, you can specify io.kubernetes.pod.namespace : backend-prod or io.kubernetes.container.name : worker-server in the container label whitelist.
Important Container labels are retrieved by running the docker inspect command. Container labels are different from Kubernetes labels. For more information, see Obtain container labels. Environment variables are the same as the environment variables that are configured to start containers. For more information, see Obtain environment variables.
|
K8s Pod Name Regular Matching | Enter the pod name. The pod name specifies the containers from which text logs are collected. Regular expression matching is supported. For example, if you specify ^(nginx-log-demo.*)$ , all containers in the pod whose name starts with nginx-log-demo are matched. |
K8s Namespace Regular Matching | Enter the namespace. The namespace specifies the containers from which text logs are collected. Regular expression matching is supported. For example, if you specify ^(default|nginx)$ , all containers in the nginx and default namespaces are matched. |
K8s Container Name Regular Matching | Enter the container name. The container name specifies the containers from which text logs are collected. Regular expression matching is supported. Kubernetes container names are defined in spec.containers. For example, if you specify ^(container-test)$ , all containers whose name is container-test are matched. |
Container Label Whitelist | Configure a container label whitelist. The whitelist specifies the containers from which text logs are collected.
Important Do not specify duplicate values for the Label Name parameter. If you specify duplicate values, only one value takes effect. If you specify a value for the Label Name parameter but do not specify a value for the Tag Value parameter, containers whose container labels contain the specified label name are matched. If you specify a value for the Label Name and Tag Value parameters, containers whose container labels contain the specified Label name:Tag value are matched. By default, string matching is performed for the values of the Tag Value parameter. Containers are matched only if the values of the container labels are the same as the values of the Tag Value parameter. If you specify a value that starts with a caret (^) and ends with a dollar sign ($) for the Tag Value parameter, regular expression matching is performed. For example, if you set the Label Name parameter to app and set the Tag Value parameter to ^(test1|test2)$ , containers whose container labels contain app:test1 or app:test2 are matched.
Key-value pairs are evaluated by using the OR operator. If a container has a container label that consists of one of the specified key-value pairs, the container is matched. |
Container Label Blacklist | Configure a container label blacklist. The blacklist specifies the containers from which text logs are not collected.
Important Do not specify duplicate values for the Label Name parameter. If you specify duplicate values, only one value takes effect. If you specify a value for the Label Name parameter but do not specify a value for the Tag Value parameter, containers whose container labels contain the specified label name are filtered out. If you specify a value for the Label Name and Tag Value parameters, containers whose container labels contain the specified Label name:Tag value are filtered out. By default, string matching is performed for the values of the Tag Value parameter. Containers are filtered out only if the values of the container labels are the same as the values of the Tag Value parameter. If you specify a value that starts with a caret (^) and ends with a dollar sign ($) for the Tag Value parameter, regular expression matching is performed. For example, if you set the Label Name parameter to app and set the Tag Value parameter to ^(test1|test2)$ , containers whose container labels contain app:test1 or app:test2 are filtered out.
Key-value pairs are evaluated by using the OR operator. If a container has a container label that consists of one of the specified key-value pairs, the container is filtered out. |
Environment Variable Whitelist | Configure an environment variable whitelist. The whitelist specifies the containers from which text logs are collected. If you specify a value for the Environment Variable Name parameter but do not specify a value for the Environment Variable Value parameter, containers whose environment variables contain the specified environment variable name are matched. If you specify a value for the Environment Variable Name and Environment Variable Value parameters, containers whose environment variables contain the specified Environment variable name:Environment variable value are matched. By default, string matching is performed for the values of the Environment Variable Value parameter. Containers are matched only if the values of the environment variables are the same as the values of the Environment Variable Value parameter. If you specify a value that starts with a caret (^) and ends with a dollar sign ($) for the Environment Variable Value parameter, regular expression matching is performed. For example, if you set the Environment Variable Name parameter to NGINX_SERVICE_PORT and set the Environment Variable Value parameter to ^(80|6379)$ , containers whose port number is 80 or 6379 are matched.
Key-value pairs are evaluated by using the OR operator. If a container has an environment variable that consists of one of the specified key-value pairs, the container is matched. |
Environment Variable Blacklist | Configure an environment variable blacklist. The blacklist specifies the containers from which text logs are not collected. If you specify a value for the Environment Variable Name parameter but do not specify a value for the Environment Variable Value parameter, containers whose environment variables contain the specified environment variable name are filtered out. If you specify a value for the Environment Variable Name and Environment Variable Value parameters, containers whose environment variables contain the specified Environment variable name:Environment variable value are filtered out. By default, string matching is performed for the values of the Environment Variable Value parameter. Containers are filtered out only if the values of the environment variables are the same as the values of the Environment Variable Value parameter. If you specify a value that starts with a caret (^) and ends with a dollar sign ($) for the Environment Variable Value parameter, regular expression matching is performed. For example, if you set the Environment Variable Name parameter to NGINX_SERVICE_PORT and set the Environment Variable Value parameter to ^(80|6379)$ , containers whose port number is 80 or 6379 are filtered out.
Key-value pairs are evaluated by using the OR operator. If a container has an environment variable that consists of one of the specified key-value pairs, the container is filtered out. |
Kubernetes Pod Label Whitelist | Configure a Kubernetes pod label whitelist. The whitelist specifies the containers from which text logs are collected. If you specify a value for the Label Name parameter but do not specify a value for the Tag Value parameter, containers whose pod labels contain the specified label name are matched. If you specify a value for the Label Name and Tag Value parameters, containers whose pod labels contain the specified Label name:Tag value are matched. By default, string matching is performed for the values of the Tag Value parameter. Containers are matched only if the values of the pod labels are the same as the values of the Tag Value parameter. If you specify a value that starts with a caret (^) and ends with a dollar sign ($) , regular expression matching is performed. For example, if you set the Label Name parameter to environment and set the Tag Value parameter to ^(dev|pre)$ , containers whose pod labels contain environment:dev or environment:pre are matched.
Key-value pairs are evaluated by using the OR operator. If a container has a pod label that consists of one of the specified key-value pairs, the container is matched. |
Kubernetes Pod Label Blacklist | Configure a Kubernetes pod label blacklist. The blacklist specifies the containers from which text logs are not collected. If you specify a value for the Label Name parameter but do not specify a value for the Tag Value parameter, containers whose pod labels contain the specified label name are filtered out. If you specify a value for the Label Name and Tag Value parameters, containers whose pod labels contain the specified Label name:Tag value are filtered out. By default, string matching is performed for the values of the Tag Value parameter. Containers are filtered out only if the values of the pod labels are the same as the values of the Tag Value parameter. If you specify a value that starts with a caret (^) and ends with a dollar sign ($) for the Tag Value parameter, regular expression matching is performed. For example, if you set the Label Name parameter to environment and set the Tag Value parameter to ^(dev|pre)$ , containers whose pod labels contain environment:dev or environment:pre are filtered out.
Key-value pairs are evaluated by using the OR operator. If a container has a pod label that consists of one of the specified key-value pairs, the container is filtered out. |
Log Tag Enrichment | Specify log tags. |
Environment Variables | After you specify environment variables as log tags, Simple Log Service adds environment variable-related fields to logs. For example, if you set the Environment Variable Name parameter to VERSION and set the Tag Name parameter to env_version , Simple Log Service adds the __tag__:__env_version__: v1.0.0 field to logs if the environment variable configurations of a container include VERSION=v1.0.0 . |
Pod Labels | After you specify pod labels as log tags, Simple Log Service adds pod label-related fields to logs. For example, if you set the Pod Label Name parameter to environment and set the Tag Name parameter to env , Simple Log Service adds the __tag__:__env__: pre field to logs if the pod label configurations of a container include environment=pre . |
File Encoding | Select the encoding format of log file. |
First Collection Size | Specify the size of data that Logtail can collect from a log file the first time Logtail collects logs from the file. The default value of First Collection Size is 1024. Unit: KB. If the file size is less than 1,024 KB, Logtail collects data from the beginning of the file. If the file size is greater than 1,024 KB, Logtail collects the last 1,024 KB of data in the file.
You can specify First Collection Size based on your business requirements. Valid values: 0 to 10485760. Unit: KB. |
Collection Blacklist | If you turn on Collection Blacklist, you must configure a blacklist to specify the directories or files that you want Simple Log Service to skip when it collects logs. You can specify exact directories and file names. You can also use wildcard characters to specify directories and file names. When you configure this parameter, you can use only asterisks (*) or question marks (?) as wildcard characters.
Important If you use wildcard characters to configure File Path and you want to skip some directories in the specified directory, you must configure Collection Blacklist and enter a complete directory. For example, if you set File Path to /home/admin/app*/log/*.log and you want to skip all subdirectories in the /home/admin/app1* directory, you must select Directory Blacklist and enter /home/admin/app1*/** in the Directory Name field. If you enter /home/admin/app1* , the blacklist does not take effect. When a blacklist is in use, computational overhead is generated. We recommend that you add up to 10 entries to the blacklist. You cannot specify a directory path that ends with a forward slash (/). For example, if you set the path to /home/admin/dir1/ , the directory blacklist does not take effect.
The following types of blacklists are supported: File Path Blacklist, File Blacklist, and Directory Blacklist. File Path BlacklistIf you select File Path Blacklist and enter /home/admin/private*.log in the File Path Name field, all files whose names are prefixed by private and suffixed by .log in the /home/admin/ directory are skipped. If you select File Path Blacklist and enter /home/admin/private*/*_inner.log in the File Path Name field, all files whose names are suffixed by _inner.log in the subdirectories whose names are prefixed by private in the /home/admin/ directory are skipped. For example, the /home/admin/private/app_inner.log file is skipped, but the /home/admin/private/app.log file is not skipped.
File BlacklistIf you select File Blacklist and enter app_inner.log in the File Name field, all files whose names are app_inner.log are skipped. Directory BlacklistIf you select Directory Blacklist and enter /home/admin/dir1 in the Directory Name field, all files in the /home/admin/dir1 directory are skipped. If you select Directory Blacklist and enter /home/admin/dir* in the Directory Name field, the files in all subdirectories whose names are prefixed by dir in the /home/admin/ directory are skipped. If you select Directory Blacklist and enter /home/admin/*/dir in the Directory Name field, all files in the dir subdirectory in each second-level subdirectory of the /home/admin/ directory are skipped. For example, the files in the /home/admin/a/dir directory are skipped, but the files in the /home/admin/a/b/dir directory are not skipped.
|
Allow File to Be Collected for Multiple Times | By default, you can use only one Logtail configuration to collect logs from a log file. To use multiple Logtail configurations to collect logs from a log file, turn on Allow File to Be Collected for Multiple Times. |