All Products
Search
Document Center

Simple Log Service:Collect container logs (standard output and files) from a cluster using the console

Last Updated:Nov 05, 2025

In Kubernetes, managing scattered container logs can be difficult, resulting in inefficient troubleshooting and high O&M costs. Deploying the Simple Log Service (SLS) LoongCollector in DaemonSet mode and configuring log collection in the SLS console allows for unified log collection and structured processing. This approach improves log retrieval, problem diagnosis, and observability analysis efficiency.

Usage notes

  • Runtime environment:

    • Supports Container Service for Kubernetes (ACK) (Managed and Dedicated editions) and self-managed Kubernetes clusters.

    • Kubernetes version 1.10.0 or later that supports Mount propagation: HostToContainer.

    • Container runtime (Docker and Containerd only)

      • Docker:

        • Requires access permissions for docker.sock.

        • Standard output collection supports only the JSON log driver.

        • Supports only the overlay and overlay2 storage drivers. For other storage driver types, you must manually mount the log directory.

      • Containerd: Requires access permissions for containerd.sock.

  • Resource requirements: LoongCollector (Logtail) runs with high priority as system-cluster-critical. Avoid deploying it if cluster resources are insufficient, as this may evict existing pods on the node.

    • CPU: Reserve at least 0.1 Core.

    • Memory: Reserve at least 150 MB for the collection component and at least 100 MB for the controller component.

    • Actual usage depends on the collection rate, the number of monitored directories and files, and the level of send congestion. Ensure that the actual usage is below 80% of the limit.

  • Permission requirements: The Alibaba Cloud account or RAM user used for deployment must have the AliyunLogFullAccess permission.

    To create custom policies, refer to the AliyunCSManagedLogRolePolicy system policy. Copy the permissions included in the policy and grant them to the target RAM user or role for fine-grained permission configuration.

Collection configuration workflow

  1. Install LoongCollector: Deploy LoongCollector in DaemonSet mode. This ensures that a collection container runs on each node in the cluster to collect logs from all containers on that node.

    For Sidecar mode, see Collect text logs from a cluster (Sidecar).
  2. Create a logstore: A logstore is a storage unit for log data. Multiple logstores can be created in a project.

  3. Create a collection configuration:

    This topic describes only common configuration parameters and core options for typical use cases. For a complete list of configuration parameters and their descriptions, see More information.

Install LoongCollector (Logtail)

LoongCollector is a new-generation log collection agent from SLS and an upgraded version of Logtail. LoongCollector and Logtail cannot coexist. To install Logtail, see Install, run, upgrade, and uninstall Logtail.

This topic describes only the basic steps to install LoongCollector. For more information about the parameters, see Install LoongCollector (Kubernetes). If you have installed LoongCollector or Logtail, skip this step and create a logstore to store the collected logs.

ACK cluster

Install LoongCollector from the Container Service for Kubernetes console. By default, logs are sent to a Simple Log Service project under the current Alibaba Cloud account.

  1. Log on to the ACK console. In the left navigation pane, click Clusters.

  2. On the Clusters page, click the name of the target cluster.

  3. In the navigation pane on the left, click Add-ons.

  4. On the Logs and Monitoring tab, find loongcollector and click Install.

    Note

    For a new cluster, on the Advanced Options section, select Enable Log Service. Then Create Project or Select Project.

    After the installation is complete, SLS automatically creates related resources in the region where the ACK cluster resides. Log on to the Simple Log Service console to view them.

    Resource type

    Resource name

    Function

    Project

    k8s-log-${cluster_id}

    A resource management unit that isolates logs of different services.

    To create a project for more flexible log resource management, see Create a project.

    Machine group

    k8s-group-${cluster_id}

    A collection of log collection nodes.

    Logstore

    config-operation-log

    Important

    Do not delete this logstore.

    Stores logs of the loongcollector-operator component. Its billing method is the same as that of a regular logstore. For more information, see Billable items for the pay-by-ingested-data mode. We recommend that you do not create collection configurations in this logstore.

Self-managed cluster

  1. Connect to the Kubernetes cluster. Select the command for your region to download LoongCollector and its dependent components:

    Regions in China:

    wget https://aliyun-observability-release-cn-shanghai.oss-cn-shanghai.aliyuncs.com/loongcollector/k8s-custom-pkg/3.0.12/loongcollector-custom-k8s-package.tgz; tar xvf loongcollector-custom-k8s-package.tgz; chmod 744 ./loongcollector-custom-k8s-package/k8s-custom-install.sh

    Regions outside China:

    wget https://aliyun-observability-release-ap-southeast-1.oss-ap-southeast-1.aliyuncs.com/loongcollector/k8s-custom-pkg/3.0.12/loongcollector-custom-k8s-package.tgz; tar xvf loongcollector-custom-k8s-package.tgz; chmod 744 ./loongcollector-custom-k8s-package/k8s-custom-install.sh
  2. Go to the loongcollector-custom-k8s-package directory and modify the ./loongcollector/values.yaml configuration file.

    # ===================== Required Information =====================
    # The name of the project where logs from this cluster will be collected, for example, k8s-log-custom-sd89ehdq
    projectName: ""
    # The region of the project, for example, Shanghai: cn-shanghai
    region: ""
    # The UID of the Alibaba Cloud account that owns the project. Enclose it in quotation marks, for example, "123456789"
    aliUid: ""
    # The network to use. Optional parameters: Internet, Intranet. The default is Internet.
    net: Internet
    # The AccessKey ID and AccessKey secret of the Alibaba Cloud account or RAM user. The account must have the AliyunLogFullAccess system policy permission.
    accessKeyID: ""
    accessKeySecret: ""
    # A custom cluster ID. The name can contain only uppercase letters, lowercase letters, digits, and hyphens (-).
    clusterID: ""
  3. In the loongcollector-custom-k8s-package directory, run the following command to install LoongCollector and other dependent components:

    bash k8s-custom-install.sh install
  4. After the installation is complete, check the running status of the components.

    If the pod fails to start, check whether the values.yaml configuration is correct and whether the relevant images were pulled successfully.
    # Check the pod status
    kubectl get po -n kube-system | grep loongcollector-ds

    SLS also automatically creates the following resources. Log on to the Simple Log Service console to view them.

    Resource type

    Resource name

    Function

    Project

    The value of projectName defined in the values.yaml file

    A resource management unit that isolates logs of different services.

    To create a project for more flexible log resource management, see Create a project.

    Machine group

    k8s-group-${cluster_id}

    A collection of log collection nodes.

    Logstore

    config-operation-log

    Important

    Do not delete this logstore.

    Stores logs of the loongcollector-operator component. Its billing method is the same as that of a regular logstore. For more information, see Pay-by-data-written billable items. We recommend that you do not create collection configurations in this logstore.

Create a logstore

If you have already created a logstore, skip this step and proceed to create a collection configuration.

SLS supports two billing methods: pay-by-ingested-data and pay-by-feature. When you create a logstore, choose the appropriate billing method as needed.
  1. Log on to the Simple Log Service console and click the name of the target project.

  2. In the navigation pane on the left, choose image Log Storage and click +.

  3. On the Create Logstore page, complete the following core configurations:

    • Logstore Name: Set a name that is unique within the project. This name cannot be changed after creation.

    • Logstore Type: Choose Standard or Query based on the specification comparison.

    • Billing Mode:

      • pay-by-feature: Billed independently for resources such as storage, indexing, and read/write operations. Suitable for small-scale use cases or when feature usage is uncertain.

      • pay-by-ingested-data: Billed only by the amount of raw data written. It provides a 30-day free storage period and free features such as data transformation and delivery. The cost model is simple and suitable for use cases where the storage period is close to 30 days or the data processing pipeline is complex.

    • Data Retention Period: Set the number of days to retain logs. The value ranges from 1 to 3650 days. 3650 days means permanent storage. The default is 30 days.

  4. Keep the default settings for other configurations and click OK. For more information about other configurations, see Manage a logstore.

Minimal configuration

After you install LoongCollector and create a logstore, a collection configuration can be set up. This section describes two basic use cases: container standard output and cluster text logs. These use cases upload raw logs to the logstore without parsing or processing, making them suitable for quickly establishing a data channel.

On the Logstore image page:

  1. Click image before the name of the target logstore.

  2. Click the image after Data Collection.

  3. Select a quick integration template based on the log source. Logs within a container have two sources:

    • Standard output (stdout and stderr): Log content printed by the container program to the console.

    • Text log file: A log file written to a specified path inside the container.

image

Collect container standard output (stdout)

  1. In the Quick Data Import dialog box, search for the template: K8S - Stdout and Stderr - New Version, then click Integrate Now.

  2. Configure the machine group, then click Next.

    • Set Scenario to Kubernetes Clusters.

    • Set Deployment Mode to ACK Daemonset or Self-managed Cluster in Daemonset Mode.

    • In the source machine group list, add the system-created machine group k8s-group-${cluster_id} to the applied machine group list on the right.

  3. Configure the Logtail, then click Next.

    • Global Configurations: Enter a configuration name.

    • Input Configurations: Enable Stdout and Stderrt or Standard Error switch. Both are enabled by default.

    • Processor Configurations:

      • (Optional) Common processing configurations: Parse raw logs into structured data using methods such as regular expressions or delimiters, or perform data masking and filtering.

      • (Optional) Other advanced configurations: Collect multiline text logs and enrich log tags to meet more fine-grained collection requirements.

  4. Query and analysis: Preview the data, click Automatic Index Generation. SLS will generate a field index. Click Next to complete the configuration.

Raw log:

10.244.0.1 - - [01/Aug/2025:10:25:30 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.88.1"

Collected raw standard output (stdout):

10.244.0.1 - - [01/Aug/2025:10:25:30 +0000] "GET / HTTP/1.1" 200 615 "-" "curl/7.88.1"

Collect cluster text logs

  1. In the Quick Data Import dialog box, search for the template: Kubernetes - File, and click Integrate Now.

  2. Configure the machine group, then click Next.

    • Set Scenario to Kubernetes Clusters.

    • Set Deployment Mode to ACK Daemonset or Self-managed Cluster in Daemonset Mode.

    • In the source machine group list, add the system-created machine group k8s-group-${cluster_id} to the applied machine group list on the right.

  3. Configure the Logtail, then click Next.

    • Global Configurations: Enter a configuration name.

    • Input Configurations:

      • Set Logtail Deployment Mode to Daemonset.

      • File Path Type:

        • Path in Container: Collect log files from within the container.

        • Host Path: Collect logs from local services on the host.

      • File Path: The absolute path for log collection. On Linux, it must start with "/".

        Example: /data/mylogs/**/*.log indicates all files with the .log extension in the /data/mylogs directory and its subdirectories.

      • Maximum Directory Monitoring Depth: The maximum directory depth matched by the wildcard character ** in the File Path. The default is 0 (current level only), and the range is 0 to 1000.

        For better performance, specify the exact directory path and setting the depth to 0.
    • Processor Configurations:

      • (Optional) Common processing configurations: Parse raw logs into structured data using methods such as regular expressions or delimiters, or perform data masking and filtering.

      • (Optional) Other advanced configurations: Collect multiline text logs and enrich log tags to meet more fine-grained collection requirements.

  4. Query and analysis: Preview the data, click Automatic Index Generation. SLS will generate a field index. Click Next to complete the configuration.

Raw log:

Aug 19 11:20:51 hostname-1 crond[2995]: (CRON) INFO (@reboot jobs will be run at computer's startup.)

The entire raw log line is stored in the content field:

content: Aug 19 11:20:51 hostname-1 crond[2995]: (CRON) INFO (@reboot jobs will be run at computer's startup.)

Common processing configurations

After completing the minimal configuration, add processing plugins in the Processor Configurations section to parse raw logs into structured data or perform data masking and filtering. To add processing plugins for an existing collection configuraiton, do as follows:

  1. In the navigation pane on the left, choose image Logstores and find the target logstore.

  2. Click image before its name to expand the logstore.

  3. Click Logtail Configurations. In the configuration list, find the target Logtail configuration and click Manage Logtail Configuration in the Actions column.

  4. On the Logtail configuration page, click Edit, scroll down to the Processor Configurations section, and configure a parsing plugin.

This section describes only native plugins that cover common log processing use cases. For more information, see Extended plugins.
Important

For Logtail 2.0 and later versions, and for the LoongCollector component, follow these plugin combination rules:

  • Prioritize using native plugins.

  • If native plugins cannot meet your needs, configure extended plugins after the native plugins.

  • Native plugins can only be used before extended plugins.

Structured configuration

Regular expression parsing

Use regular expressions to extract log fields and parse the log into key-value pairs.

  1. Add Sample Log: Configuring a log sample helps you configure log processing parameters and simplifies the configuration process.

  2. Click Add Processor and choose Native Processor > Data Parsing (Regex Mode):

    • Regular Expression: Used to match logs. Generate it automatically or enter it manually:

      • Automatic generation:

        • Click Generate.

        • In the Log Sample, select the log content that you want to extract.

        • Click Generate Regular Expression.

          image

      • Manual input: Manually Enter Regular Expression based on the log format.

      After configuration, click Validate to test if the regular expression can correctly parse the log content.

    • Extracted Field: Set the corresponding field names (Key) for the extracted log content (Value).

Raw log:

127.0.0.1 - - [16/Aug/2024:14:37:52 +0800] "GET /wp-admin/admin-ajax.php?action=rest-nonce HTTP/1.1" 200 41 "http://www.example.com/wp-admin/post-new.php?post_type=page" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0"

Custom regular expression parsing: Regular expression (\S+)\s-\s(\S+)\s\[([^]]+)]\s"(\w+)\s(\S+)\s([^"]+)"\s(\d+)\s(\d+)\s"([^"]+)"\s"([^"]+).*.

body_bytes_sent: 41
http_referer: http://www.example.com/wp-admin/post-new.php?post_type=page
http_user_agent: Mozilla/5.0 (Windows NT 10.0; Win64; ×64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36 Edg/127.0.0.0
remote_addr: 127.0.0.1
remote_user: -
request_method: GET
request_protocol: HTTP/1.1
request_uri: /wp-admin/admin-ajax.php?action=rest-nonce
status: 200
time_local: 16/Aug/2024:14:37:52 +0800

Delimiter-based parsing

Use a delimiter to structure the log content and parse it into multiple key-value pairs. Both single-character and multi-character delimiters are supported.

Click Add Processor and choose Native Processor Data Parsing (Delimiter Mode):

  • Delimiter: Specify the character used to split the log content.

    Example: For a CSV file, select Custom and enter a comma (,).

  • Quote: If a field value contains the separator, you need to specify a quote to enclose the field to avoid incorrect splitting.

  • Extracted Field: Set the corresponding field name (Key) for each column in order of separation. The rules are as follows:

    • Field names can contain only letters, digits, and underscores (_).

    • Must start with a letter or an underscore (_).

    • Maximum length: 128 bytes.

Raw log:

05/May/2025:13:30:28,10.10.*.*,"POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=****************&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=******************************** HTTP/1.1",200,18204,aliyun-sdk-java

Split fields by the specified character ,:

ip:10.10.*.*
request:POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=****************&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=******************************** HTTP/1.1
size:18204
status:200
time:05/May/2025:13:30:28
user_agent:aliyun-sdk-java

Standard JSON parsing

Structure an object-type JSON log and parse it into key-value pairs.

Click Add Processor and choose Native Processor > Data Parsing (JSON Mode):

  • Original Field: The default value is content. This field stores the raw log content to be parsed.

  • Keep the default settings for other configurations.

Raw Log:

{"url": "POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=U0Ujpek********&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=pD12XYLmGxKQ%2Bmkd6x7hAgQ7b1c%3D HTTP/1.1", "ip": "10.200.98.220", "user-agent": "aliyun-sdk-java", "request": {"status": "200", "latency": "18204"}, "time": "05/Jan/2025:13:30:28"}

Standard JSON key-value pairs are automatically extracted:

ip: 10.200.98.220
request: {"status": "200", "latency" : "18204" }
time: 05/Jan/2025:13:30:28
url: POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=U0Ujpek******&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=pD12XYLmGxKQ%2Bmkd6x7hAgQ7b1c%3D HTTP/1.1
user-agent:aliyun-sdk-java

Nested JSON parsing

Parse a nested JSON log into key-value pairs by specifying the expansion depth.

Click Add Processor and choose Extended Processor > Expand JSON Field:

  • Original Field: The source field to be expanded, for example, content.

  • JSON Expansion Depth: The expansion level of the JSON object. 0 means fully expanded (default), 1 means the current level, and so on.

  • Character to Concatenate Expanded Keys: The connector for field names during JSON expansion. The default is an underscore _.

  • Name Prefix of Expanded Keys: Specify the prefix for field names after JSON expansion.

  • Expand Array: Enable this to expand an array into key-value pairs with indexes.

    Example: {"k":["a","b"]} is expanded to {"k[0]":"a","k[1]":"b"}.

    To rename the expanded fields (for example, from `prefix_s_key_k1` to `new_field_name`), add a Rename Fields plugin afterward to perform the renaming.

Raw log:

{"s_key":{"k1":{"k2":{"k3":{"k4":{"k51":"51","k52":"52"},"k41":"41"}}}}}

Expansion depth: 0, with the expansion depth used as a prefix.

0_s_key_k1_k2_k3_k41:41
0_s_key_k1_k2_k3_k4_k51:51
0_s_key_k1_k2_k3_k4_k52:52

Expansion depth: 1, with the expansion depth used as a prefix.

1_s_key:{"k1":{"k2":{"k3":{"k4":{"k51":"51","k52":"52"},"k41":"41"}}}}

JSON array parsing

Use the json_extract function to extract JSON objects from a JSON array.

Set Processing Method to SPL:

  • SPL Statement: Use the json_extract function to extract JSON objects from a JSON array.

    Example: Extract elements from the JSON array in the log field content and store the results in new fields json1 and json2.

    * | extend json1 = json_extract(content, '$[0]'), json2 = json_extract(content, '$[1]')

Raw log:

[{"key1":"value1"},{"key2":"value2"}]

Extracted JSON array structure:

json1:{"key1":"value1"}
json2:{"key2":"value2"}

NGINX log parsing

Structure the log content into multiple key-value pairs based on the definition in log_format. If the default content does not meet your needs, use a custom format.

Click Add Processor and choose Native Processor > Data Parsing (NGINX Mode):

  • NGINX Log Configuration: Copy the complete log_format definition from your NGINX server configuration file (usually located at /etc/nginx/nginx.conf) and paste it into this text box.

    Example:

    log_format main  '$remote_addr - $remote_user [$time_local] "$request" ''$request_time $request_length ''$status $body_bytes_sent "$http_referer" ''"$http_user_agent"';
    Important

    The format definition here must be exactly the same as the format that generates the logs on the server. Otherwise, log parsing will fail.

Raw log:

192.168.*.* - - [15/Apr/2025:16:40:00 +0800] "GET /nginx-logo.png HTTP/1.1" 0.000 514 200 368 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.*.* Safari/537.36"

Parsed into key-value pairs based on the log_format main definition:

body_bytes_sent: 368
http_referer: -
http_user_agent : Mozi11a/5.0 (Nindows NT 10.0; Win64; x64) AppleMebKit/537.36 (KHTML, like Gecko) Chrome/131.0.x.x Safari/537.36
remote_addr:192.168.*.*
remote_user: -
request_length: 514
request_method: GET
request_time: 0.000
request_uri: /nginx-logo.png
status: 200
time_local: 15/Apr/2025:16:40:00

Apache log parsing

Structure the log content into multiple key-value pairs based on the definition in the Apache log configuration file.

Click Add Processor and choose Native Processor > Data Parsing (Apache Mode):

  • Set Log Format to combined.

  • APACHE LogFormat Configuration: The system automatically fills in the configuration based on the Log Format.

    Important

    Make sure to verify the auto-filled content to ensure it is exactly the same as the LogFormat defined in the Apache configuration file on the server (usually located at /etc/apache2/apache2.conf).

Raw log:

1 192.168.1.10 - - [08/May/2024:15:30:28 +0800] "GET /index.html HTTP/1.1" 200 1234 "https://www.example.com/referrer" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.X.X Safari/537.36"

Apache Common Log Format combined parsing:

http_referer:https://www.example.com/referrer
http_user_agent:Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.X.X Safari/537.36
remote_addr:192.168.1.10
remote_ident:-
remote_user:-
request_method:GET
request_protocol:HTTP/1.1
request_uri:/index.html
response_size_bytes:1234
status:200
time_local:[08/May/2024:15:30:28 +0800]

Data masking

Mask sensitive data in logs.

In the Processor Configurations section, click Add Processor and choose Native Processor > Data Masking:

  • Original Field: The source field that stores the log content before parsing.

  • Data Masking Method:

    • const: Replace sensitive content with a specified string.

    • md5: Replace sensitive content with its corresponding MD5 hash.

  • Replacement String: When you set Data Masking Method to const, you need to enter a string to replace the sensitive content.

  • Content Expression that Precedes Replaced Content: Used to find sensitive content. Configure it using RE2 syntax.

  • Content Expression to Match Replaced Content: The expression for the sensitive content. Configure it using RE2 syntax.

Raw log:

[{'account':'1812213231432969','password':'04a23f38'}, {'account':'1812213685634','password':'123a'}]

Masking result:

[{'account':'1812213231432969','password':'********'}, {'account':'1812213685634','password':'********'}]

Content filtering

Match log field values based on regular expressions and collect only logs that meet the whitelist conditions.

In the Processor Configurations section, click Add Processor and choose Native Processor > Data Filtering:

  • Field Name: The log field to be filtered.

  • Field Value: The regular expression used for filtering. Only full matching is supported. Partial keyword matching is not supported.

Raw log:

{"level":"WARNING","timestamp":"2025-09-23T19:11:40+0800","cluster":"yilu-cluster-0728","message":"Disk space is running low","freeSpace":"15%"}
{"level":"ERROR","timestamp":"2025-09-23T19:11:42+0800","cluster":"yilu-cluster-0728","message":"Failed to connect to database","errorCode":5003}
{"level":"INFO","timestamp":"2025-09-23T19:11:47+0800","cluster":"yilu-cluster-0728","message":"User logged in successfully","userId":"user-123"}

Filtered log: Set Field Name to level and Field Value to WARNING|ERROR. This means only logs where the level field value is WARNING or ERROR are collected.

{"level":"WARNING","timestamp":"2025-09-23T19:11:40+0800","cluster":"yilu-cluster-0728","message":"Disk space is running low","freeSpace":"15%"}
{"level":"ERROR","timestamp":"2025-09-23T19:11:42+0800","cluster":"yilu-cluster-0728","message":"Failed to connect to database","errorCode":5003}

Time parsing

Parse the time field in the log and set the parsing result as the log's __time__ field.

In the Processor Configurations section, click Add Processor and choose Native Processor > Time Parsing:

  • Original Field: The source field that stores the log content before parsing.

  • Time Format: Set the corresponding time format based on the time content in the log.

  • Time Zone: Select the time zone of the log's time field. By default, the machine's time zone is used, which is the time zone of the environment where the LoongCollector (Logtail) process is running.

Raw log:

{"level":"INFO","timestamp":"2025-09-23T19:11:47+0800","cluster":"yilu-cluster-0728","message":"User logged in successfully","userId":"user-123"}

Time parsing:

image

Other advanced configurations

After completing the minimal configuration or common processing configurations, perform the following operations to collect multiline logs, configure topic types, and more for fine-grained log collection.

To modify an existing collection configuration, do as follows:

  1. In the navigation pane on the left, choose image Logstores and find the target logstore.

  2. Click image before its name to expand the logstore.

  3. Click Logtail Configurations. In the configuration list, find the target Logtail configuration and click Manage Logtail Configuration in the Actions column.

  4. On the Logtail configuration page, click Edit.

Feature

Purpose

Multiline log collection

Process cross-line logs such as Java stack traces.

Log topic types

Set topics based on machine groups, file paths, or custom rules to categorize your log sources.

Container filtering and blacklist

  • Filter by container name, label, or environment variable.

  • Ignore sensitive or irrelevant log files.

Log tag enrichment

Add environment variables and container information as metadata fields.

Compression for log transmission

Enable lz4/zstd to reduce bandwidth.


Multiline log collection

By default, SLS works in single-line mode, treating each line of text as a separate log. This can incorrectly split multiline logs containing content such as stack traces or JSON, leading to a loss of context.

To address this issue, enable Multi-line Mode and define a Regex to Match First Line. This allows SLS to accurately identify the starting line of a complete log, thereby merging multiple lines into a single log entry.

Processor Configurations:

  • Enable Multi-line Mode.

  • Set Type to Custom or Multi-line JSON.

    • Custom: The format of the raw log is not fixed. You need to configure Regex to Match First Line to identify the starting line of each log.

      • Regex to Match First Line: Generate it automatically or enter it manually. The regular expression must match a complete line of data. For example, the matching regular expression in the example above is \[\d+-\d+-\w+:\d+:\d+,\d+]\s\[\w+]\s.*.

        • Automatic generation: Click Generate. Then, in the Log Sample text box, select the log content to be extracted and click Generate Regular Expression.

        • Manual input: Click Manually Enter Regular Expression. After entering, click Validate.

    • Multi-line JSON: Select this when the raw logs are all in standard JSON format. SLS will automatically handle line breaks within a single JSON log.

  • Processing Method If Splitting Fails:

    • Discard: If a piece of text does not match the first-line rule, it is discarded.

    • Retain Single Line: Unmatched text is processed as individual single-line logs.

Raw log:

[2023-10-01T10:30:01,000] [INFO] java.lang.Exception: exception happened
    at TestPrintStackTrace.f(TestPrintStackTrace.java:3)
    at TestPrintStackTrace.g(TestPrintStackTrace.java:7)
    at TestPrintStackTrace.main(TestPrintStackTrace.java:16)

Single-line mode: Each line is a separate log, and the stack information is broken up, losing context.

image

Multi-line mode: A first-line regular expression identifies the complete log, preserving the full semantic structure.

content:[2023-10-01T10:30:01,000] [INFO] java.lang.Exception: exception happened
    at TestPrintStackTrace.f(TestPrintStackTrace.java:3)
    at TestPrintStackTrace.g(TestPrintStackTrace.java:7)
    at TestPrintStackTrace.main(TestPrintStackTrace.java:16)

Configure topic types

Global Configurations > Other Global Configurations > Log Topic Type: Select the topic generation method.

  • Machine Group Topic: SLS lets you apply one collection configuration to multiple machine groups. When LoongCollector reports data, it uses the machine group's topic as the log topic and uploads it to the logstore. Use topics to distinguish logs from different machine groups.

  • File Path Extraction: If different users or applications write logs to different top-level directories but with the same subdirectory paths and filenames, it becomes difficult to distinguish the log source from the filename. Configure File Path Extraction. Use a regular expression to match the full file path and use the matched result (username or application name) as the log topic to be uploaded to the logstore.

    Note

    In the regular expression for the file path, you must escape the forward slash (/).

    Extract using a file path regular expression

    Use case: Different users record logs in different directories, but the log filenames are the same. The directory paths are as follows.

    /data/logs
    ├── userA
    │   └── serviceA
    │       └── service.log
    ├── userB
    │   └── serviceA
    │       └── service.log
    └── userC
        └── serviceA
            └── service.log

    If you only configure the file path as /data/logs and the filename as service.log in the Logtail Configuration, LoongCollector (Logtail) will collect the content from all three service.log files into the same logstore. This makes it impossible to distinguish which user produced which log. Use a regular expression to extract values from the file path to generate different log topics.

    Regular expression

    Extraction result

    \/data\/logs\/(.*)\/serviceA\/.*
    __topic__: userA
    __topic__: userB
    __topic__: userC

    Extract using multiple capturing groups

    Use case: If a single log topic is not enough to distinguish the source of the logs, configure multiple regular expression capturing groups in the log file path to extract key information. These capturing groups include named capturing groups (?P<name>) and unnamed capturing groups.

    • Named capturing group: The generated tag field is __tag__:{name}.

    • Unnamed capturing group: The generated tag field is __tag__:__topic_{i}__, where {i} is the sequence number of the capturing group.

    Note

    When there are multiple capturing groups in the regular expression, the __topic__ field is not generated.

    For example, if the file path is /data/logs/userA/serviceA/service.log, extract multiple values from the file path in the following ways:

    Example

    Regular expression

    Extraction result

    Use an unnamed capturing group for regular expression extraction.

    \/data\/logs\/(.*?)\/(.*?)\/service.log
    __tag__:__topic_1__: userA
    __tag__:__topic_2__: serviceA

    Use a named capturing group for regular expression extraction.

    \/data\/logs\/(?P<user>.*?)\/(?P<service>.*?)\/service.log
    __tag__:user: userA
    __tag__:service: serviceA

    Validation: After configuration, query logs based on the log topic.

    On the log query and analysis page, enter the corresponding generated log topic, such as __topic__: userA or __tag__:__topic_1__: userA, to query logs for that topic.

    image

  • Custom: Enter customized:// + custom topic name to use a custom static log topic.


Container filtering and blacklists

Container filtering

Input Configurations:

  • Enable Container Filtering, click Add, select a filtering method, and configure it. Multiple conditions are combined with an "AND" relationship.

    • Environment Variable Blacklist/Whitelist: Specify the environment variable conditions for the containers to be collected.

    • K8s Pod Label Blacklist/Whitelist: Specify the label conditions for the pod where the containers to be collected are located.

    • K8s Pod Name Regex Match: Specify the containers to be collected by pod name.

    • K8s Namespace Regex Match: Specify the containers to be collected by namespace name.

    • K8s Container Name Regex Match: Specify the containers to be collected by container name.

    • Container Label Blacklist/Whitelist: Collect containers whose labels meet the conditions. Used in Docker use cases. Not recommended for Kubernetes use cases.

All regular expression matching uses the Go language's RE2 engine, which has a limited feature set compared to engines such as PCRE. To write regular expressions, refer to Appendix: Regular expression limits (container filtering) for the correct syntax.

Blacklist

Input Configurations > Other Input Configurations: Enable Collection Blacklist, click Add, and configure the blacklist.

Supports full matching and wildcard matching for directories and filenames. Wildcard characters only support the asterisk (*) and the question mark (?).
  • File Path Blacklist: The file path to be ignored. Example:

    • /home/admin/private*.log: During collection, ignore all files in the /home/admin/ directory that start with "private" and end with ".log".

    • /home/admin/private*/*_inner.log: During collection, ignore files ending with "_inner.log" within directories that start with "private" under the /home/admin/ directory.

  • File Blacklist: Configure the filenames to be ignored during collection. Example:

    • app_inner.log: During collection, ignore all files named app_inner.log.

  • Directory Blacklist: The directory path cannot end with a forward slash (/). Example:

    • /home/admin/dir1/: The directory blacklist will not take effect.

    • /home/admin/dir*: During collection, ignore files in subdirectories that start with "dir" under the /home/admin/ directory.

    • /home/admin/*/dir: During collection, ignore all files in subdirectories named "dir" at the second level under the /home/admin/ directory. For example, files in the /home/admin/a/dir directory are ignored, while files in the /home/admin/a/b/dir directory are collected.


Log tag enrichment

Input Configurations > Log Tag Enrichment:

  • Enable Log Tag Enrichment and click Add:

    • Environment Variables: Configure an environment variable name and a tag name. The environment variable value will be stored in the tag name.

      • Environment Variable Name: Specify the name of the environment variable to be extracted.

      • Tag Name: The name of the environment variable tag.

    • Pod Labels: Configure a pod label name and a tag name. The pod label value will be stored in the tag name.

      • Pod Label Name: The name of the Kubernetes pod label to be extracted.

      • Tag Name: The name of the tag.


Compression for log transmission

Output Configurations:

Note

Only Logtail 1.3.4 and later versions support zstd compression.

  • lz4: Fast compression speed, lower compression ratio.

  • zstd: High compression ratio, slightly lower speed, high memory usage.

What to do next

  1. Log query and analysis: Each collected container text log contains the following default field information:

    Field

    Description

    __tag__:__hostname__

    The name of the container's host.

    __tag__:__path__

    The path of the log file within the container.

    __tag__:_container_ip_

    The IP address of the container.

    __tag__:_image_name_

    The name of the image used by the container.

    __tag__:_pod_name_

    The name of the pod.

    __tag__:_namespace_

    The namespace to which the pod belongs.

    __tag__:_pod_uid_

    The unique identifier (UID) of the pod.

  2. Data visualization: Use visualization dashboards to monitor key metric trends.

  3. Automatic alerting for data anomalies: Set alert policies to detect system anomalies in real time.

Troubleshooting: No data collected from container logs

  1. Check for new logs: After configuring LoongCollector (Logtail) for collection, if there are no new logs in the target log file, LoongCollector (Logtail) will not collect from that file.

2. View LoongCollector (Logtail) runtime logs

View the runtime logs of LoongCollector (Logtail) itself to obtain detailed error information.

  1. Log on to the Logtail container:

    1. Query the Logtail pod.

      kubectl get po -n kube-system | grep logtail

      The system will return a result similar to the following.

      logtail-ds-****d                                             1/1       Running    0          8d
      logtail-ds-****8                                             1/1       Running    0          8d
    2. Log on to the pod.

      kubectl exec -it -n kube-system logtail-ds-****d -- bash

      Here, logtail-ds-****d is the pod ID. Replace it with the actual value.

  1. View Logtail runtime logs:

    Logtail logs are stored in the /usr/local/ilogtail/ directory within the Logtail container. The filenames are ilogtail.LOG and logtail_plugin.LOG. Log on to the Logtail container and run the following commands to view the log files:

    Go to the /usr/local/ilogtail/ directory.
    cd /usr/local/ilogtail
    
    View the ilogtail.LOG and logtail_plugin.LOG files.
    cat ilogtail.LOG
    cat logtail_plugin.LOG

    Purpose: Check the alarm type in the error logs and find the corresponding solution in Common errors in Simple Log Service data collection.

3. Check the machine group heartbeat

Check the machine group heartbeat status: Go to the image Resources > Machine Groups page, click the name of the target machine group. In the Machine Group Configurations > Machine Group Status section, view the heartbeat status and record the number of nodes with an OK heartbeat status.

  1. Check the number of worker nodes in the container cluster.

    1. Obtain the cluster KubeConfig and connect to the cluster using kubectl.

    2. View the number of worker nodes in the cluster.

      kubectl get node | grep -v master

      The system will return a result similar to the following.

      NAME                                 STATUS    ROLES     AGE       VERSION
      cn-hangzhou.i-bp17enxc2us3624wexh2   Ready     <none>    238d      v1.10.4
      cn-hangzhou.i-bp1ad2b02jtqd1shi2ut   Ready     <none>    220d      v1.10.4
  2. Compare whether the number of nodes with an OK heartbeat status is consistent with the number of worker nodes in the container cluster. Choose a troubleshooting method based on the comparison result.

    • The heartbeat status of all nodes in the machine group is Failed:

      • If it is a self-managed cluster, check if the following parameters are configured correctly: {regionId}, {aliuid}, {access-key-id}, and {access-key-secret}.

        If they are incorrect, run the helm del --purge alibaba-log-controller command to delete the installation package, and then reinstall.

    • The number of nodes with an OK heartbeat status is less than the number of worker nodes in the cluster.

      • Determine if a DaemonSet has been manually deployed using a YAML file.

        1. Run the following command. If there is a result, it means a DaemonSet was previously deployed manually using a YAML file.

          kubectl get po -n kube-system -l k8s-app=logtail
        2. Download the latest version of the DaemonSet template.

        3. Configure parameters such as ${your_region_name}, ${your_aliyun_user_id}, and ${your_machine_group_name} based on actual values.

        4. Update the resource.

          kubectl apply -f ./logtail-daemonset.yaml

4. Check the collection configuration filter conditions

In the Simple Log Service console, check the Logtail collection configuration. Pay close attention to whether the IncludeLabel, ExcludeLabel, IncludeEnv, ExcludeEnv, and other settings in the Logtail configuration meet your collection requirements.

  • The label here refers to the container label, which is the label in Docker inspect, not the label in Kubernetes.

  • Temporarily remove the IncludeLabel, ExcludeLabel, IncludeEnv, and ExcludeEnv configurations to see if logs can be collected normally. If so, it indicates a problem with the configuration of these parameters.

FAQ

How do I send ACK cluster logs to a project in another Alibaba Cloud account?

Manually install LoongCollector (Logtail) in the ACK cluster and configure it with the target Alibaba Cloud account ID or AccessKey. This enables sending container logs to an SLS project in another Alibaba Cloud account.

Use case: Collect log data from an ACK cluster to an SLS project in a different Alibaba Cloud account for reasons such as organizational structure, permission isolation, or unified monitoring. Manually install LoongCollector (Logtail) for cross-account configuration.

Procedure: The following procedure uses the manual installation of LoongCollector as an example. To learn how to install Logtail, see Install and configure Logtail.

  1. Connect to the Kubernetes cluster. Select the command for your region to download LoongCollector and its dependent components:

    Regions in China:

    wget https://aliyun-observability-release-cn-shanghai.oss-cn-shanghai.aliyuncs.com/loongcollector/k8s-custom-pkg/3.0.12/loongcollector-custom-k8s-package.tgz; tar xvf loongcollector-custom-k8s-package.tgz; chmod 744 ./loongcollector-custom-k8s-package/k8s-custom-install.sh

    Regions outside China:

    wget https://aliyun-observability-release-ap-southeast-1.oss-ap-southeast-1.aliyuncs.com/loongcollector/k8s-custom-pkg/3.0.12/loongcollector-custom-k8s-package.tgz; tar xvf loongcollector-custom-k8s-package.tgz; chmod 744 ./loongcollector-custom-k8s-package/k8s-custom-install.sh
  2. Go to the loongcollector-custom-k8s-package directory and modify the ./loongcollector/values.yaml configuration file.

    # ===================== Required Information =====================
    # The name of the project where logs from this cluster will be collected, for example, k8s-log-custom-sd89ehdq
    projectName: ""
    # The region of the project, for example, Shanghai: cn-shanghai
    region: ""
    # The UID of the Alibaba Cloud account that owns the project. Enclose it in quotation marks, for example, "123456789"
    aliUid: ""
    # The network to use. Optional parameters: Internet, Intranet. The default is Internet.
    net: Internet
    # The AccessKey ID and AccessKey secret of the Alibaba Cloud account or RAM user. The account must have the AliyunLogFullAccess system policy permission.
    accessKeyID: ""
    accessKeySecret: ""
    # A custom cluster ID. The name can contain only uppercase letters, lowercase letters, digits, and hyphens (-).
    clusterID: ""
  3. In the loongcollector-custom-k8s-package directory, run the following command to install LoongCollector and other dependent components:

    bash k8s-custom-install.sh install
  4. After the installation is complete, check the running status of the components.

    If the pod fails to start, check whether the values.yaml configuration is correct and whether the relevant images were pulled successfully.
    # Check the pod status
    kubectl get po -n kube-system | grep loongcollector-ds

    SLS also automatically creates the following resources. Log on to the Simple Log Service console to view them.

    Resource type

    Resource name

    Function

    Project

    The value of projectName defined in the values.yaml file

    A resource management unit that isolates logs of different services.

    To create a project for more flexible log resource management, see Create a project.

    Machine group

    k8s-group-${cluster_id}

    A collection of log collection nodes.

    Logstore

    config-operation-log

    Important

    Do not delete this logstore.

    Stores logs of the loongcollector-operator component. Its billing method is the same as that of a regular logstore. For more information, see Pay-by-data-written billable items. We recommend that you do not create collection configurations in this logstore.

How can I collect logs from the same file or container standard output using multiple collection configurations?

By default, to prevent data duplication, SLS restricts each log source to be collected by only one collection configuration:

  • A text log file can only match one Logtail collection configuration.

  • A container's standard output (stdout) can also only be collected by one standard output collection configuration.

  1. Log on to the Simple Log Service console and go to the target project.

  2. In the navigation pane on the left, choose image Logstores and find the target logstore.

  3. Click image before its name to expand the logstore.

  4. Click Logtail Configurations. In the configuration list, find the target Logtail configuration and click Manage Logtail Configuration in the Actions column.

  5. On the Logtail configuration page, click Edit and scroll down to the Input Configurations section:

    • To collect text file logs: Enable Allow File to Be Collected for Multiple Times.

    • To collect container standard output: Enable Allow Collection by Different Logtail Configurations.

Appendix: Regular expression limits (container filtering)

The regular expressions for container filtering are based on the Go RE2 engine. The RE2 engine has several syntax limitations compared to other engines, such as Perl Compatible Regular Expressions (PCRE). Note the following when you write regular expressions:

1. Named group syntax differences

Go uses the (?P<name>...) syntax to define named groups and does not support the (?<name>...) syntax from PCRE.

  • Correct: (?P<year>\d{4})

  • Incorrect: (?<year>\d{4})

2. Unsupported regular expression features

The following common but complex regular expression features are not supported by RE2. Avoid using them:

  • Assertions: (?=...), (?!...), (?<=...), (?<!...)

  • Conditional expressions: (?(condition)true|false)

  • Recursive matching: (?R), (?0)

  • Subprogram references: (?&name), (?P>name)

  • Atomic groups: (?>...)

3. Recommendations

Use a tool such as Regex101 to debug regular expressions. To validate compatibility, select the Golang (RE2) mode. The plugin cannot parse or match expressions that contain unsupported syntax.

More information

Global configuration parameters

Parameter

Description

Configuration Name

The name of the Logtail configuration. It must be unique within its project. The name cannot be changed after the Logtail configuration is created.

Topic Type

Select the method for generating the log topic. Options include Machine Group Topic, File Path Extraction, and Custom.

Advanced Parameters

Other optional advanced feature parameters related to the global configuration. For more information, see CreateLogtailPipelineConfig.

Input configuration parameters

Parameter

Description

Logtail Deployment Mode

DaemonSet: Deploys one LoongCollector on each node of the cluster to collect logs from all containers on that node.

Sidecar: Each pod runs a LoongCollector container to collect logs from all containers within that pod. Log collection for different pods is isolated.

File Path Type

Supports the following options:

  • Path in Container: Select this to collect text log files from within a container.

  • Host Path: Select this to collect service logs from the cluster nodes.

File Path

Set the log directory and filename based on the log's location on the host (such as an ECS instance).

  • If the target host is a Linux system, the log path must start with a forward slash (/), for example, /apsara/nuwa/**/app.Log.

  • If the target host is a Windows system, the log path must start with a drive letter, for example, C:\Program Files\Intel\**\*.Log.

Both directory and file names support full and wildcard modes. For filename rules, see Wildcard matching. The log path wildcard character supports only the asterisk (*) and the question mark (?).

The log path supports multi-level directory matching. This means all files that meet the criteria in the specified directory and its subdirectories are collected. For example:

  • /apsara/nuwa/**/*.log indicates files with the .log extension in the /apsara/nuwa directory and its recursive subdirectories.

  • /var/logs/app_*/**/*.log indicates files with the .log extension in all directories matching the app_* format under the /var/logs directory and their recursive subdirectories.

  • /var/log/nginx/**/access* indicates files starting with access in the /var/log/nginx directory and its recursive subdirectories.

Max Directory Monitoring Depth

Set the maximum depth for monitoring the log directory. This is the maximum directory depth matched by the wildcard character ** in the File Path. 0 represents monitoring only the current directory.

Stdout and Stderr

After enabling Stdout and Stderr, Logtail will collect the container's standard output.

Standard Error

After enabling Standard Error, Logtail will collect the container's standard error.

Allow Collection by Different Logtail Configurations

By default, a container's standard output log can only match one new-version Logtail standard output collection configuration. If the standard output needs to be collected by multiple new-version standard output collection configurations, you must enable Allow Collection by Different Logtail Configurations.

Enable Container Metadata Preview

After enabling Enable Container Metadata Preview, view container metadata after creating the Logtail configuration. This includes matched container information and full container information.

Container Filtering

  • Filter condition description

Important
  • Container Label refers to the Label in Docker inspect, not the Label in Kubernetes. To retrieve it, see Obtain Container Label.

  • Environment Variable refers to the environment variable information configured during container startup. To retrieve it, see Obtain Container Environment Variable.

  • In Kubernetes use cases, we recommend using Kubernetes-level information (such as K8s Pod Name Regex Match, K8s Namespace Regex Match, K8s Container Name Regex Match, K8s Pod Label Whitelist) for container filtering.

  1. The Namespace and container name in Kubernetes are mapped to container Labels, specifically io.kubernetes.pod.namespace and io.kubernetes.container.name. We recommend using these two container Labels for container filtering. For example, if a pod belongs to the namespace backend-prod and the container name is worker-server, set the container label whitelist to io.kubernetes.pod.namespace : backend-prod or io.kubernetes.container.name : worker-server to collect logs from that container.

  2. If these two container labels do not meet your filtering needs, use the environment variable blacklist and whitelist for container filtering.

K8s Pod Name Regex Match

Specify the containers to be collected by pod name. Regular expression matching is supported. For example, setting it to ^(nginx-log-demo.*)$ matches all containers in pods whose names start with nginx-log-demo.

K8s Namespace Regex Match

Specify the containers to be collected by namespace name. Regular expression matching is supported. For example, setting it to ^(default|nginx)$ matches all containers in the nginx and default namespaces.

K8s Container Name Regex Match

Specify the containers to be collected by container name (the Kubernetes container name is defined in spec.containers). Regular expression matching is supported. For example, setting it to ^(container-test)$ matches all containers named container-test.

Container Label Whitelist

The container Label whitelist specifies the containers to be collected. It is empty by default, which means all container standard outputs are collected. To set a container Label whitelist, the LabelKey is required, and the LabelValue is optional.

  • If LabelValue is empty, all containers with a Label that includes the LabelKey will be matched.

  • If LabelValue is not empty, only containers with a Label that matches LabelKey=LabelValue will be matched.

    By default, LabelValue is a string match, meaning it only matches if the LabelValue is exactly the same as the container's Label value. If the value starts with ^ and ends with $, it is a regular expression match. For example, if you set LabelKey to io.kubernetes.container.name and LabelValue to ^(nginx|cube)$, it will match containers named nginx or cube.

Multiple whitelist entries have an OR relationship, meaning a container is matched if its Label satisfies any of the whitelist entries.

Container Label Blacklist

The container Label blacklist excludes containers from collection. It is empty by default, which means no containers are excluded. To set a container Label blacklist, the LabelKey is required, and the LabelValue is optional.

  • If LabelValue is empty, all containers with a Label that includes the LabelKey will be excluded.

  • If LabelValue is not empty, only containers with a Label that matches LabelKey=LabelValue will be excluded.

    By default, LabelValue is a string match, meaning it only matches if the LabelValue is exactly the same as the container's Label value. If the value starts with ^ and ends with $, it is a regular expression match. For example, if you set LabelKey to io.kubernetes.container.name and LabelValue to ^(nginx|cube)$, it will match containers named nginx or cube.

Multiple blacklist entries have an OR relationship, meaning a container is excluded if its Label satisfies any of the blacklist entries.

Environment Variable Whitelist

The environment variable whitelist specifies the containers to be collected. It is empty by default, which means all container standard outputs are collected. To set an environment variable whitelist, the EnvKey is required, and the EnvValue is optional.

  • If EnvValue is empty, all containers with an environment variable that includes the EnvKey will be matched.

  • If EnvValue is not empty, only containers with an environment variable that matches EnvKey=EnvValue will be matched.

    By default, EnvValue is a string match, meaning it only matches if the EnvValue is exactly the same as the environment variable's value. If the value starts with ^ and ends with $, it is a regular expression match. For example, if you set EnvKey to NGINX_SERVICE_PORT and EnvValue to ^(80|6379)$, it will match containers with a service port of 80 or 6379.

Multiple whitelist entries have an OR relationship, meaning a container is matched if its environment variables satisfy any of the key-value pairs.

Environment Variable Blacklist

The environment variable blacklist excludes containers from collection. It is empty by default, which means no containers are excluded. To set an environment variable blacklist, the EnvKey is required, and the EnvValue is optional.

  • If EnvValue is empty, logs from all containers with an environment variable that includes the EnvKey will be excluded.

  • If EnvValue is not empty, only containers with an environment variable that matches EnvKey=EnvValue will be excluded.

    By default, EnvValue is a string match, meaning it only matches if the EnvValue is exactly the same as the environment variable's value. If the value starts with ^ and ends with $, it is a regular expression match. For example, if you set EnvKey to NGINX_SERVICE_PORT and EnvValue to ^(80|6379)$, it will match containers with a service port of 80 or 6379.

Multiple blacklist entries have an OR relationship, meaning a container is excluded if its environment variables satisfy any of the key-value pairs.

K8s Pod Label Whitelist

Specify the containers to be collected using a Kubernetes Label whitelist. To set a Kubernetes Label whitelist, the LabelKey is required, and the LabelValue is optional.

  • If LabelValue is empty, all containers with a Kubernetes Label that includes the LabelKey will be matched.

  • If LabelValue is not empty, only containers with a Kubernetes Label that matches LabelKey=LabelValue will be matched.

    By default, LabelValue is a string match, meaning it only matches if the LabelValue is exactly the same as the Kubernetes Label's value. If the value starts with ^ and ends with $, it is a regular expression match. For example, if you set LabelKey to app and LabelValue to ^(test1|test2)$, it will match containers with Kubernetes Labels app:test1 or app:test2.

Multiple whitelist entries have an OR relationship, meaning a container is matched if its Kubernetes Label satisfies any of the whitelist entries.

Note
  • Changing a Label on a Kubernetes management resource (such as a Deployment) at runtime does not restart the specific worker pod. Therefore, the pod cannot perceive this change, which may cause the matching rule to fail. When setting K8s Label blacklists and whitelists, use the Kubernetes Label in the pod as the standard. For more information about Kubernetes Labels, see Labels and Selectors.

K8s Pod Label Blacklist

Exclude containers from collection using a Kubernetes Label blacklist. To set a Kubernetes Label blacklist, the LabelKey is required, and the LabelValue is optional.

  • If LabelValue is empty, all containers with a Kubernetes Label that includes the LabelKey will be excluded.

  • If LabelValue is not empty, only containers with a Kubernetes Label that matches LabelKey=LabelValue will be excluded.

    By default, LabelValue is a string match, meaning it only matches if the LabelValue is exactly the same as the Kubernetes Label's value. If the value starts with ^ and ends with $, it is a regular expression match. For example, if you set LabelKey to app and LabelValue to ^(test1|test2)$, it will match containers with Kubernetes Labels app:test1 or app:test2.

Multiple blacklist entries have an OR relationship, meaning a container is excluded if its Kubernetes Label satisfies any of the blacklist entries.

Note
  • Changing a Label on a Kubernetes management resource (such as a Deployment) at runtime does not restart the specific worker pod. Therefore, the pod cannot perceive this change, which may cause the matching rule to fail. When setting K8s Label blacklists and whitelists, use the Kubernetes Label in the pod as the standard. For more information about Kubernetes Labels, see Labels and Selectors.

Log Tag Enrichment

Add environment variables and Kubernetes labels to logs as log tags.

Environment Variables

After setting environment variable extension fields, SLS will add environment variable-related fields to the logs. For example, if you set Environment Variable Name to VERSION and Tag Name to env_version, and the container includes the environment variable VERSION=v1.0.0, this information will be added to the log as the field __tag__:__env_version__: v1.0.0.

Pod Labels

After setting Kubernetes pod extension fields, SLS will add Kubernetes pod-related fields to the logs. For example, if you set Pod Label Name to app and Tag Name to k8s_pod_app, and the Kubernetes pod includes the Label app=serviceA, this information will be added to the log as the field __tag__:__k8s_pod_app__: serviceA.

File Encoding

Select the encoding format of the log file.

First Collection Size

When the configuration first takes effect, this is the size from the end of the file from which collection starts. The default initial collection size is 1024 KB. The value range is 0 to 10485760, in KB.

  • For the first collection, if the file is smaller than 1024 KB, collection starts from the beginning of the file.

  • For the first collection, if the file is larger than 1024 KB, collection starts from 1024 KB from the end of the file.

Collection Blacklist

After enabling the Collection Blacklist switch, configure a blacklist to ignore specified directories or files during collection. It supports full matching and wildcard matching for directories and filenames. The wildcard characters supported are the asterisk (*) and the question mark (?).

Important
  • If you use a wildcard character when configuring the File Path but need to filter out some of those paths, you must enter the corresponding full path in the Collection Blacklist to ensure the blacklist configuration takes effect.

    For example, if you set File Path to /home/admin/app*/log/*.log but want to filter all subdirectories under /home/admin/app1*, you must select Directory Blacklist and configure the directory as /home/admin/app1*/**. If you configure it as /home/admin/app1*, the blacklist will not take effect.

  • Matching against a blacklist has a computational overhead. We recommend keeping the number of blacklist entries within 10.

  • A directory path cannot end with a forward slash (/). For example, if you set the path to /home/admin/dir1/, the directory blacklist will not take effect.

It supports setting by file path blacklist, file blacklist, and directory blacklist. The details are as follows:

File Path Blacklist

  • Select File Path Blacklist and configure the path as /home/admin/private*.log. This will ignore all files in the /home/admin/ directory that start with "private" and end with ".log" during collection.

  • Select File Path Blacklist and configure the path as /home/admin/private*/*_inner.log. This will ignore files ending with "_inner.log" within directories that start with "private" under the /home/admin/ directory during collection. For example, the file /home/admin/private/app_inner.log is ignored, while the file /home/admin/private/app.log is collected.

File Blacklist

Select File Blacklist and configure the filename as app_inner.log. This will ignore all files named app_inner.log during collection.

Directory Blacklist

  • Select Directory Blacklist and configure the directory as /home/admin/dir1. This will ignore all files in the /home/admin/dir1 directory during collection.

  • Select Directory Blacklist and configure the directory as /home/admin/dir*. This will ignore all files in subdirectories that start with "dir" under the /home/admin/ directory during collection.

  • Select Directory Blacklist and configure the directory as /home/admin/*/dir. This will ignore all files in subdirectories named "dir" at the second level under the /home/admin/ directory during collection. For example, files in the /home/admin/a/dir directory are ignored, while files in the /home/admin/a/b/dir directory are collected.

Allow File to Be Collected for Multiple Times

By default, a log file can only match one Logtail configuration. If the logs in a file need to be collected multiple times, enable Allow File to Be Collected for Multiple Times.

Advanced Parameters

Other optional advanced feature parameters related to the file input plugin. For more information, see CreateLogtailPipelineConfig.

Processor configuration parameters

Parameter

Description

Log Sample

A sample of the log to be collected. Use a log from your actual scenario. The log sample helps configure log processing parameters and reduces configuration difficulty. You can add multiple samples, with a total length not exceeding 1500 characters.

[2023-10-01T10:30:01,000] [INFO] java.lang.Exception: exception happened
    at TestPrintStackTrace.f(TestPrintStackTrace.java:3)
    at TestPrintStackTrace.g(TestPrintStackTrace.java:7)
    at TestPrintStackTrace.main(TestPrintStackTrace.java:16)

Multi-line Mode

  • Type: A multiline log is one where each log entry is distributed across multiple consecutive lines. It is necessary to distinguish each log entry from the log content.

    • Custom: Use a Regex to Match First Line to distinguish each log entry.

    • Multi-line JSON: Each JSON object is expanded into multiple lines, for example:

      {
        "name": "John Doe",
        "age": 30,
        "address": {
          "city": "New York",
          "country": "USA"
        }
      }
  • Processing Method If Splitting Fails:

    Exception in thread "main" java.lang.NullPointerException
        at com.example.MyClass.methodA(MyClass.java:12)
        at com.example.MyClass.methodB(MyClass.java:34)
        at com.example.MyClass.main(MyClass.java:½0)

    For the log content above, if SLS fails to split it:

    • Discard: Discards this log segment

    • Retain Single Line: Retains each line of log text as a separate log entry, resulting in a total of four log entries.

Processing Method

Processors, which includes native and extended plugins. For more information, see Overview of Logtail plugins for data processing.

Important

For limitations on the use of processing plugins, refer to the prompts on the console page.

  • Logtail 2.0:

    • Native plugins can be combined in any way.

    • Native and extended plugins can be used at the same time, but extended plugins can only appear after all native plugins.

  • Logtail versions earlier than 2.0:

    • Adding both native and extended plugins at the same time is not supported.

    • Native plugins can only be used to collect text logs. When using native plugins, the following requirements must be met:

      • The first plugin must be regular expression parsing, delimiter-based parsing, JSON parsing, NGINX pattern parsing, Apache pattern parsing, or IIS pattern parsing.

      • From the second to the last plugin, there can be at most one time parsing plugin, one filtering plugin, and multiple data masking plugins.

    • For the Retain Original Field if Parsing Fails and Retain Original Field if Parsing Succeeds parameters, only the following combinations are valid. Other combinations are invalid.

      • Upload only successfully parsed logs:

        image

      • Upload parsed logs on success, and upload raw logs on failure:

        image

      • On success, upload parsed logs and append the raw log field. On failure, upload raw logs.

        For example, if the raw log "content": "{"request_method":"GET", "request_time":"200"}" is parsed successfully, appending the raw field adds another field to the parsed log. The field name is New Name of Original Field (if not filled, it defaults to the original field name), and the field value is the raw log {"request_method":"GET", "request_time":"200"}.

        image