1. Pull the image

On a host machine that has Docker installed, run the following command to pull the LoongCollector image. Replace ${region_id} with the region ID of the region that hosts the host machine or of a nearby region, such as cn-hangzhou, to improve download speed and stability.

# LoongCollector image address
docker pull aliyun-observability-release-registry.${region_id}.cr.aliyuncs.com/loongcollector/loongcollector:v3.0.12.0-25723a1-aliyun
# Logtail image address
docker pull registry.${region_id}.aliyuncs.com/log-service/logtail:v2.1.11.0-aliyun

2. Start the LoongCollector container

Run the following command to start the container. Make sure that you mount the required directories and set the required environment variables:

docker run -d \
    -v /:/logtail_host:ro \
    -v /var/run/docker.sock:/var/run/docker.sock \
    --env ALIYUN_LOGTAIL_CONFIG=/etc/ilogtail/conf/${sls_upload_channel}/ilogtail_config.json \
    --env ALIYUN_LOGTAIL_USER_ID=${aliyun_account_id} \
    --env ALIYUN_LOGTAIL_USER_DEFINED_ID=${user_defined_id} \
    aliyun-observability-release-registry.${region_id}.cr.aliyuncs.com/loongcollector/loongcollector:v3.0.12.0-25723a1-aliyun

Parameters:

• ${aliyun_account_id}: The ID of your Alibaba Cloud account.

• ${user_defined_id}: The custom identifier of the machine group, which binds the container to the machine group, such as user-defined-docker-1. The identifier must be unique within the region.

• ${sls_upload_channel}: The log upload channel. The value consists of the region that hosts the project and the network transmission type, as described in the following table.

For the network characteristics behind each transmission type, see the Appendix: Network transmission types section of this topic.

Required startup conditions:

• Correctly configure the three key environment variables: ALIYUN_LOGTAIL_CONFIG, ALIYUN_LOGTAIL_USER_ID, and ALIYUN_LOGTAIL_USER_DEFINED_ID.

• Mount /var/run/docker.sock to listen for container lifecycle events.

• Mount / to /logtail_host to access the file system of the host machine.

3. Verify the container status

Run the following command:

docker ps | grep loongcollector

The following output is returned:

6ad510001753   aliyun-observability-release-registry.cn-beijing.cr.aliyuncs.com/loongcollector/loongcollector:v3.0.12.0-25723a1-aliyun   "/usr/local/ilogtail…"   About a minute ago   Up About a minute             recursing_shirley

4. Configure the machine group

In the left-side navigation pane, choose Resource Group > Machine Groups. Click > Create Machine Group, configure the following parameters, and then click OK:

• Name: Enter a custom name for the machine group, such as docker-host-group.

• Machine Group Identifier: Select Custom Identifier.

• Custom Identifier: Enter the ${user_defined_id} that you set when you started the container. The values must be identical. Otherwise, the association fails.

5. Verify the heartbeat of the machine group

Click the name of the new machine group to go to its details page, and check the Machine Group Status:

• OK: LoongCollector is connected to Simple Log Service.

• FAIL: See Summary of heartbeat issues to troubleshoot the issue.

1. Global and input configuration

Before you begin, make sure that you have selected a data import template and bound a machine group. This step defines the name of the collection configuration, the log source, and the collection scope.

2. Log processing and structuring

Configure log processing rules to convert raw unstructured logs into structured data and improve log query and analysis efficiency. We recommend that you add a log sample first:

On the Logtail Configuration page, in the Processor Configurations section, click Add Sample Log and enter the log content that you want to collect. Simple Log Service identifies the log format from the sample and helps generate regular expressions and parsing rules, which simplifies the configuration.

Scenario 1: Process multi-line logs, such as Java stack logs

Logs such as Java exception stacks and JSON usually span multiple lines. In the default collection mode, they are split into several incomplete entries and the context is lost. To prevent this, turn on multi-line mode and configure a first-line regular expression to merge consecutive lines of the same log into one complete entry.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, turn on Multi-line Mode:

• Type: Select Custom or Multi-line JSON.

  • Custom: The format of the raw logs is not fixed. Configure a Regex to Match First Line to identify the starting line of each log.

    • Regex to Match First Line: You can generate the expression automatically or enter it manually. The regular expression must match a complete line. In the preceding example, the expression is \[\d+-\d+-\w+:\d+:\d+,\d+]\s\[\w+]\s.*.

      • Automatic generation: Click Automatically Generate Regular Expression, select the log content that you want to extract in the Log Sample text box, and then click Generate Regular Expression.

      • Manual input: Click Manually Enter Regular Expression. After you enter the expression, click Validate.

  • Multi-line JSON: If all raw logs are in standard JSON format, Simple Log Service automatically handles the line breaks inside a single JSON log.

• Processing Method If Splitting Fails:

  • Discard: If a text segment does not match the first-line rule, the segment is discarded.

  • Retain Single Line: Unmatched text is split and retained in the original single-line mode.

Scenario 2: Structure logs

When raw logs are unstructured or semi-structured text, such as NGINX access logs or application output logs, querying and analyzing them directly is inefficient. Simple Log Service provides multiple data parsing plugins that automatically convert raw logs in different formats into structured data, which provides a solid data foundation for subsequent analysis, monitoring, and alerting.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section:

• Add a parsing plugin: Click Add Processor and configure a plugin such as regular expression parsing, delimiter parsing, or JSON parsing based on the actual format. For the parameters of each plugin, see the Appendix: Native parsing plugins section of this topic. The following example collects NGINX logs, so select Native Processor > Data Parsing (NGINX Mode).

  • NGINX Log Configuration: Copy the completelog_format definition from the NGINX server configuration file (nginx.conf) and paste it into this text box. Example:

    log_format main  '$remote_addr - $remote_user [$time_local] "$request" ''$request_time $request_length ''$status $body_bytes_sent "$http_referer" ''"$http_user_agent"';

    exactly the same as the format that generates the logs on the server. Otherwise, log parsing fails.

• General configuration parameters: The following parameters appear in multiple data parsing plugins and have the same function and usage.

  • Source field: The name of the source field to parse. Default value: content, which is the entire collected log content.

  • Keep source field on parse failure: We recommend that you turn on this option. If a plugin cannot parse a log, for example, because of a format mismatch, this option keeps the complete raw log content in the specified source field instead of dropping it.

  • Keep source field on parse success: If you select this option, the raw log content is retained even when the log is parsed successfully.

3. Log filtering

Collecting large volumes of low-value or irrelevant logs, such as DEBUG or INFO logs, wastes storage resources, increases costs, degrades query efficiency, and creates a data leak risk. Use fine-grained filtering policies for efficient and secure log collection.

Filtering is configured in two places in the console: content filtering is a processing plugin in the Processor Configurations section, whereas the collection blacklist and container filtering are switches in the Input Configurations section.

4. Log categorization

When multiple applications or instances share the same log format, log sources are hard to tell apart, which leaves queries without context and reduces analysis efficiency. To address this, configure topics and log tagging for automated context association and logical categorization.

/data/logs
├── userA
│   └── serviceA
│       └── service.log
├── userB
│   └── serviceA
│       └── service.log
└── userC
    └── serviceA
        └── service.log

5. Output configuration

By default, all logs are sent to the current LogStore with lz4 compression. To distribute logs from the same source to different LogStores, perform the following steps.

Dynamic distribution to multiple targets

• Sending to multiple targets requires LoongCollector 3.0.0 or later. Logtail does not support this feature.

• You can configure a maximum of five output targets.

• After you configure multiple output targets, the collection configuration no longer appears in the collection configuration list of the current LogStore. To view, modify, or delete a multi-target distribution configuration, see How do I manage multi-target distribution configurations?.

  Procedure: On the Logtail Configuration page, in the Output Configurations section:

1. Click to expand the output configuration.

2. Click Add Output Targets and complete the following settings:

   • Logstores: Select the destination LogStore.

   • Compression Method: lz4 and zstd are supported.

   • Route Settings: Routes and distributes logs based on their tag fields. Logs that meet the routing configuration are uploaded to the destination LogStore. If the routing configuration is empty, all collected logs are uploaded to the destination LogStore.

     • Tag Name: The name of the tag field used for routing. Enter the field name directly, such as __path__, without the __tag__: prefix. Tag fields fall into the following two categories:

       • Agent-related: Related to the collection agent itself and independent of plugins, such as __hostname__ and __user_defined_id__.

       • Input plugin-related: Provided by an input plugin, which enriches the logs with the related information, such as __path__ for file collection and _pod_name_ and _container_name_ for Kubernetes collection.

         For more information about tags, see Manage collection tags.

     • Tag Value: Logs whose tag field value matches this value are sent to the destination LogStore.

     • Discard this tag?: If you turn on this switch, the uploaded logs do not contain this tag field.

View uploaded logs

• Confirm that the log file has new content: LoongCollector collects only incremental logs. Run tail -f /path/to/your/log/file and trigger an application operation to make sure that new logs are being written.

• Query logs: Go to the query and analysis page of the destination LogStore and click Search & Analyze. The default time range is the last 15 minutes. Check whether new logs are arriving. By default, each collected Docker container text log contains the following fields:

For the metadata field names that container standard output uses, see the Appendix: Comparison of the new and old container standard output versions section of this topic.

Troubleshoot common issues

View the running status of LoongCollector (Logtail)

Run the following command:

docker exec ${logtail_container_id} /etc/init.d/ilogtaild status

View the version, IP address, start time, and other information of LoongCollector (Logtail)

Run the following command:

docker exec ${logtail_container_id} cat /usr/local/ilogtail/app_info.json

View the operational logs of LoongCollector (Logtail)

The operational logs of LoongCollector (Logtail) are stored in the /usr/local/ilogtail/ directory inside the container. The file name is ilogtail.LOG, and rotated files are compressed and stored as ilogtail.LOG.x.gz. Example:

# View the operational logs of LoongCollector

docker exec a287de895e40 tail -n 5 /usr/local/ilogtail/loongcollector.LOG

# View the operational logs of Logtail

docker exec a287de895e40 tail -n 5 /usr/local/ilogtail/ilogtail.LOG

The following output is returned:

[2025-08-25 09:17:44.610496]    [info]  [22]    /build/loongcollector/file_server/polling/PollingModify.cpp:75          polling modify resume:succeeded
[2025-08-25 09:17:44.610497]    [info]  [22]    /build/loongcollector/file_server/polling/PollingDirFile.cpp:100                polling discovery resume:starts
[2025-08-25 09:17:44.610498]    [info]  [22]    /build/loongcollector/file_server/polling/PollingDirFile.cpp:103                polling discovery resume:succeeded
[2025-08-25 09:17:44.610499]    [info]  [22]    /build/loongcollector/file_server/FileServer.cpp:117            file server resume:succeeded
[2025-08-25 09:17:44.610500]    [info]  [22]    /build/loongcollector/file_server/EventDispatcher.cpp:1019              checkpoint dump:succeeded

Restart LoongCollector (Logtail)

Run the following commands:

# Stop LoongCollector

docker exec a287de895e40 /etc/init.d/ilogtaild stop

# Start LoongCollector

docker exec a287de895e40 /etc/init.d/ilogtaild start

Common error messages

Error log: The parameter is invalid : uuid=none

Problem description: The LoongCollector (Logtail) log file (/usr/local/ilogtail/ilogtail.LOG) contains the error log The parameter is invalid : uuid=none.

Solution: On the host machine, create a product_uuid file, enter any valid UUID in it, such as 169E98C9-ABC0-4A92-B1D2-AA6239C0D261, and mount the file to the /sys/class/dmi/id/product_uuid directory in the LoongCollector (Logtail) container.

How do I collect the same log file or container standard output with multiple collection configurations?

By default, to prevent duplicate data, Simple Log Service allows each log source to be collected by only one collection configuration:

• A text log file can be matched by only one Logtail collection configuration.

• Container standard output (stdout):

• If you use the new standard output template, the standard output can be collected by only one standard output collection configuration by default.

• If you use the old standard output template, collection by multiple configurations is supported by default and requires no extra configuration.

  To allow the same log source to be collected by multiple collection configurations, perform the following steps:

1. Log on to the Simple Log Service console and go to the destination project.

2. In the left-side navigation pane, choose LogStores and find the destination LogStore.

3. Click the icon next to its name to expand the LogStore.

4. Click Logtail Configuration, find the destination Logtail configuration in the configuration list, and click Manage Logtail Configuration in the Actions column.

5. On the Logtail configuration page, click Edit and scroll down to the Input Configurations section: - To collect text file logs: Turn on Allow File to Be Collected for Multiple Times. - To collect container standard output: Turn on Allow Collection by Different Logtail Configurations.

How do I manage multi-target distribution configurations?

Because a multi-target distribution configuration is associated with multiple LogStores, you must maintain this type of configuration on the project-level management page:

1. Log on to the Simple Log Service console and click the name of the destination project.

2. On the destination project page, in the left-side navigation pane, click Resource Group > Configurations.

   This page centrally manages all collection configurations in the project, including configurations left behind after a LogStore was deleted by mistake.

Regular expression parsing

Extracts log fields with a regular expression and parses the log into key-value pairs so that each field can be queried and analyzed independently.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, click Add Processor and select Native Processor > Data Parsing (Regex Mode):

• Regular Expression: The expression used to match logs. You can generate it automatically or enter it manually:

• Automatic generation:

  1. Click Automatically Generate Regular Expression.

  2. In the Log Sample section, highlight the log content that you want to extract.

  3. Click Generate Regular Expression.

     Confirm that log content in the correct format is pasted in the Log Sample section, such as an Apache Combined access log, and then click Generate Regular Expression in that section to automatically generate the parsing expression. - Manual input: Manually enter a regular expression based on the log format.

  After you finish the configuration, click Validate to test whether the regular expression parses the log content correctly. - Extracted Field: Set a field name (Key) for each piece of extracted log content (Value). - For information about the other parameters, see the general configuration parameters described in Scenario 2: Structure logs.

Delimiter parsing

Structures log content with a delimiter and parses it into multiple key-value pairs. Single-character and multi-character delimiters are supported.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, click Add Processor and select Native Processor > Data Parsing (Delimiter Mode):

• Delimiter: Specify the character used to split the log content. Example: For a CSV file, select Custom and enter a comma (,).

• Quote: If a field value contains the delimiter, specify a quote character to enclose the field and prevent incorrect splitting.

• Extracted Field: Set a field name (Key) for each column in the order in which the columns are split. The following rules apply:

• A field name can contain only letters, digits, and underscores (_).

• A field name must start with a letter or an underscore (_).

• Maximum length: 128 bytes.

• For information about the other parameters, see the general configuration parameters described in Scenario 2: Structure logs.

Standard JSON parsing

Structures an Object-type JSON log and parses it into key-value pairs.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, click Add Processor and select Native Processor > Data Parsing (JSON Mode):

• Original Field: Default value: content. This field stores the raw log content to be parsed.

• For information about the other parameters, see the general configuration parameters described in Scenario 2: Structure logs.

Nested JSON parsing

Parses a nested JSON log into key-value pairs by specifying an expansion depth.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, click Add Processor and select Extended Processor > Expand JSON Field:

• Original Field: The name of the original field to expand, such as content.

• JSON Expansion Depth: The expansion level of the JSON object. A value of 0 indicates full expansion (default value), a value of 1 indicates the current level, and so on.

• Character to Concatenate Expanded Keys: The character used to concatenate field names during JSON expansion. Default value: underscore (_).

• Name Prefix of Expanded Keys: The prefix of the field names after JSON expansion.

• Expand Array: Turn on this option to expand an array into key-value pairs with indexes. Example: {"k":["a","b"]} is expanded to {"k[0]":"a","k[1]":"b"}.

  To rename an expanded field, for example, to change prefix_s_key_k1 to new_field_name, add a Rename Fields plugin afterwards to complete the mapping.

• For information about the other parameters, see the general configuration parameters described in Scenario 2: Structure logs.

JSON array parsing

Use the json_extractfunction to extract JSON objects from a JSON array.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, switch Processing Method to SPL, configure the SPL statement, and use the json_extract function to extract JSON objects from the JSON array.

Example: Extract the elements of the JSON array from the content log field and store the results in the new json1 and json2 fields.

* | extend json1 = json_extract(content, '$[0]'), json2 = json_extract(content, '$[1]')

Apache log parsing

Structures log content based on the definitions in the Apache log configuration file and parses it into multiple key-value pairs.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, click Add Processor and select Native Processor > Data Parsing (Apache Mode):

• Log Format: combined

• APACHE LogFormat Configuration: Simple Log Service automatically populates this setting based on Log Format. Check the automatically populated content to make sure that it is identical to the LogFormat defined in the Apache configuration file on your server, which is usually located at /etc/apache2/apache2.conf.

• For information about the other parameters, see the general configuration parameters described in Scenario 2: Structure logs.

Data masking

Masks sensitive data in logs.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, click Add Processor and select Native Processor > Data Masking:

• Original Field: The original field that stores the log content before parsing.

• Data Masking Method:

  • const: Replaces the sensitive content with the specified string.

  • md5: Replaces the sensitive content with its MD5 value.

• Replacement String: If you set Data Masking Method to const, enter the string used to replace the sensitive content.

• Content Expression that Precedes Replaced Content: Locates the sensitive content. Configure it with the RE2 syntax.

• Content Expression to Match Replaced Content: The expression of the sensitive content. Configure it with the RE2 syntax.

Time parsing

Parses the time field in the log and sets the parsing result as the __time__ field of the log.

Example:

Procedure: On the Logtail Configuration page, in the Processor Configurations section, click Add Processor and select Native Processor > Time Parsing:

• Original Field: The original field that stores the log content before parsing.

• Time Format: Set the matching time format based on the time content in the log.

• Time Zone: Select the time zone of the time field in the log. The machine time zone is used by default, which is the time zone of the environment that hosts the LoongCollector (Logtail) process.

Core advantages of the new standard output

• Significantly improved performance

• Refactored in C++, the new version delivers a 180% to 300% performance improvement over the previous Go implementation.

• It supports data processing with native plugins and multi-threading parallel processing, which fully utilizes system resources.

• It supports flexible combinations of native plugins and Go plugins to meet the needs of complex scenarios.

• Stronger reliability

• It supports a rotation queue for standard output logs and unifies the log collection mechanism with the file collection mechanism, which provides high reliability when standard output logs rotate rapidly.

• Lower resource consumption

• CPU usage is reduced by 20% to 25%.

• Memory usage is reduced by 20% to 25%.

• Enhanced O&M consistency

• Unified parameter configuration: The configuration parameters of the new standard output collection plugin are consistent with those of the file collection plugin.

• Unified metadata management: The naming of container metadata fields and the tag storage location are unified with the file collection scenario, so the consumer side maintains only one set of processing logic.

Feature comparison of the new and old versions

Container metadata field mapping

In the new version, all metadata fields are stored in the tag section of the log in the __tag__:<key> format instead of being embedded in the log content.

Impact of the new version on users

• Consumer-side adaptation: Because the storage location changes from the content to the tag, you must adjust your log consumption logic accordingly. For example, you must access the fields through __tag__ when you run a query.

• SQL compatibility: Query SQL is already automatically compatible with both versions, so you do not need to modify your query statements to process logs from either version.