Container Service for Kubernetes:Collect container logs (stdout/stderr or files) from Kubernetes clusters using the console

Check that logs are arriving

1. Confirm new log entries exist: LoongCollector only collects incremental logs. Run the following command and trigger a business operation to generate new log entries:

   tail -f /path/to/your/log/file

2. Query logs in the console: Go to the Logstore's query and analysis page and click Search & Analyze. The default time range is the last 15 minutes. To quickly verify collection, run this query to check for recent log entries:

   Field name	Description
   __tag__:__hostname__	Name of the container's host.
   __tag__:__path__	Path of the log file in the container.
   __tag__:_container_ip_	IP address of the container.
   __tag__:_image_name_	Name of the container image.
   __tag__:_pod_name_	Name of the pod.
   __tag__:_namespace_	Namespace of the pod.
   __tag__:_pod_uid_	Unique identifier (UID) of the pod.

   * | SELECT __time__, __tag__:_pod_name_, __tag__:_namespace_, content LIMIT 20

   To filter by a specific pod:

   * | SELECT * WHERE __tag__:_pod_name_ LIKE 'your-pod-name%' LIMIT 20

   Each collected container text log includes the following fields by default:

Troubleshooting

Manage multi-destination distribution configurations

Multi-destination configurations are associated with multiple Logstores, so manage them at the project level:

1. Log on to the Simple Log Service console and click the target project.

2. In the left navigation pane, click Resource Group > Configurations.

This page centralizes all collection configurations in the project, including configurations that remain after their Logstores were accidentally deleted.

Send ACK cluster logs to a project in a different Alibaba Cloud account

Manually install LoongCollector in the ACK cluster and configure it with the destination account's ID and AccessKey credentials. This sends logs to a Simple Log Service project under a different Alibaba Cloud account—useful for organizational separation, permission isolation, or centralized monitoring.

Follow the self-managed cluster installation steps above, setting aliUid, accessKeyID, and accessKeySecret to the values for the destination account.

For information about installing Logtail instead, see Logtail installation and configuration.

Allow multiple configurations to collect the same log source

By default, each log source can only match one collection configuration:

• A text log file can match only one Logtail configuration.

• A container's stdout can only be collected by one new-template configuration by default. (Legacy-template configurations support multiple collection copies without extra configuration.)

To allow multiple configurations to collect the same source:

1. Log on to the Simple Log Service console. Go to the target project.

2. In the left navigation pane, select Logstores, find the target Logstore, and click to expand it.

3. Click Logtail Configuration, find the target configuration, and click Manage Logtail Configuration in the Actions column.

4. On the configuration page, click Edit and scroll to Input Configurations:

   • For text log files: enable Allow File to Be Collected for Multiple Times.

   • For container stdout: enable Allow Collection by Different Logtail Configurations.

Dependency error when uninstalling the loongcollector (logtail-ds) component in ACK

Problem: Uninstalling the loongcollector (logtail-ds) component fails with:

Dependencies of addons are not met: terway-eniip depends on logtail-ds(>0.0) whose version is v3.x.x.x-aliyun or will be v3.x.x.x-aliyun

Cause: The terway-eniip network plugin has log collection enabled and depends on loongcollector (logtail-ds). Remove the dependency before uninstalling:

1. Log on to the ACK console.Container Service for Kubernetes consoleSimple Log Service console

2. Click the target cluster name and go to its details page.

3. In the left navigation pane, click Add-ons.

4. Find terway-eniip and click Disable Logging.

5. Click OK in the dialog box.

6. After the change takes effect, uninstall the loongcollector (logtail-ds) component.

Why is the last log segment delayed or truncated?

Log truncation occurs when a log file lacks a trailing newline or a multi-line log is not fully written before collection.

Behavior by version:

• Versions earlier than 1.8: LoongCollector waits for the next write to trigger output, which may delay the last log entry indefinitely.

• Versions 1.8 and later: A timeout mechanism automatically submits incomplete content after 60 seconds (default). Adjust this value if needed, but do not set it to 0—doing so causes truncation.

To extend the flush timeout:

1. Log on to the Simple Log Service console. Go to the target project and Logstore.

2. Find the target Logtail configuration and click Manage Logtail Configuration.

3. Click Edit, then go to Input Configurations > Other Input Configurations > Advanced Parameters.

4. Add the following JSON:

   {
     "FlushTimeoutSecs": 1
   }

   The default value is determined by the default_reader_flush_timeout startup parameter (usually a few seconds). Set the value to at least 1 second. The unit is seconds.

5. Click OK.

Why does LoongCollector switch from private network to public network during runtime?

When LoongCollector detects abnormal private network communication (network unavailability or connection timeouts), it automatically switches to the public network endpoint to prevent log loss.

• LoongCollector: Automatically switches back to the private network when connectivity recovers.

• Logtail: Does not switch back automatically. Restart Logtail manually to restore private network communication.

Regular expression parsing

Extract log fields using regular expressions and parse logs into key-value pairs.

Example:

Configuration: Click Add Processor and select Native Processor > Data Parsing (Regex Mode):

• Regular expression: Match logs using a regular expression.

  • Auto-generate: click Generate Regular Expression automatically, select the content to extract in Log Sample, and click Generate Regular Expression. See screenshot:

  • Manual input: click Manually enter a regular expression, enter your expression, then click Validate.

• Extracted field: Map each captured group to a field name (Key).

• For shared parameters, see Scenario 2: Structured logs.

Delimiter parsing

Split log fields by a separator character and map each column to a field name.

Example (comma-delimited):

Configuration: Click Add Processor and select Native Processor > Data Parsing (Delimiter Mode):

• Delimiter: The character used to split log fields. For CSV files, select Custom and enter a comma (,).

• Quote: Enclose field values containing a delimiter to prevent incorrect splitting.

• Extracted field: Map each column to a field name in order. Field names can only contain letters, numbers, and underscores, must start with a letter or underscore, and have a maximum length of 128 bytes.

• For shared parameters, see Scenario 2: Structured logs.

Standard JSON parsing

Parse flat JSON logs into key-value pairs.

Example:

Configuration: Click Add Processor and select Native Processor > Data Parsing (JSON Mode):

• Original field: The field containing the raw JSON. Default is content.

• For shared parameters, see Scenario 2: Structured logs.

Nested JSON parsing

Flatten deeply nested JSON objects into key-value pairs by specifying an expansion depth.

Example:

Configuration: Click Add Processor and select Extended Processor > Expand JSON Field:

• Original field: The source field to expand (for example, content).

• JSON expansion depth: 0 means full expansion (default); 1 means the top level only.

• Character to concatenate expanded keys: Separator for expanded field names. Default is underscore (_).

• Name prefix of expanded keys: Prefix for all expanded field names.

• Expand array: Enable to expand arrays into indexed key-value pairs. For example, {"k":["a","b"]} expands to {"k[0]":"a","k[1]":"b"}.

To rename expanded fields, add a Rename Fields plugin after this one.

JSON array parsing

Extract JSON objects from a JSON array using the json_extract function.

Example:

Configuration: In the Logtail Configuration area, switch the Processing Method to SPL and enter an SPL statement using the json_extract function.

Example: Extract elements from the content field array and store them in json1 and json2:

* | extend json1 = json_extract(content, '$[0]'), json2 = json_extract(content, '$[1]')

Apache log parsing

Parse Apache access logs using the format definition from your Apache configuration file.

Example:

Configuration: Click Add Processor and select Native Processor > Data Parsing (Apache Mode):

• Log format: Select combined.

• APACHE LogFormat configuration: The system auto-populates based on the selected format.

Data masking

Mask sensitive values in logs before they are stored.

Example:

Configuration: Click Add Processor and select Native Processor > Data Masking:

• Original field: The field containing the content to mask.

• Data masking method:

  • const: Replace sensitive content with a fixed string.

  • md5: Replace sensitive content with its MD5 hash.

• Replacement string: The replacement string when using the const method.

• Content expression that precedes replaced content: Identifies the context before the sensitive content. Uses RE2 syntax.

• Content expression to match replaced content: Matches the sensitive content. Uses RE2 syntax.

Time parsing

Parse a time field and set the result as the __time__ field for the log.

Example:

Raw log	Parsed result
{"level":"INFO","timestamp":"2025-09-23T19:11:47+0800","message":"User logged in"}

Configuration: Click Add Processor and select Native Processor > Time Parsing:

• Original field: The field containing the raw time value.

• Time format: The format string matching your log's time content. See Time formats.

• Time zone: The time zone of the log time field. Defaults to the time zone of the machine running LoongCollector.

Core advantages of the new template

Metadata storage format

Field mapping

Query SQL has been automatically updated for compatibility, so existing query statements work with both old and new format logs simultaneously.