Simple Log Service:Manage LoongCollector collection tags

Limitations

• This feature requires LoongCollector 3.0.10 or later. If you use Logtail or an earlier version of LoongCollector, upgrade to the latest version.

• If your log consumption process depends on the modified tags, you may encounter compatibility issues.

Tag categories

A tag consists of a key-value pair, similar to a field index. For example, in __tag__:__inode__:263554, __tag__:__inode__ is the key (tag name) and 263554 is the value.

Tags fall into two categories based on their source:

• Agent-related: Associated with the collection agent itself and independent of plugins. Examples include IP addresses and hostnames.

• Input plugin-related: Added by input plugins to enrich logs with contextual information, such as file inodes, read offsets, pod names, namespaces, and image names.

Billing

Tags are free of charge. However, if you configure a tag as a field index, charges apply. For more information about billing, see Billing. For more information about how to create and delete a field index, see Create an index.

Create or modify a tag name

LoongCollector provides a tag processing feature. You can configure advanced parameters to add, delete, and rename tags.

Delete a tag

To delete a tag, set its value to an empty string in the advanced parameter configuration.

In the following example, the value of AGENT_TAG is set to an empty string to remove the tag. For more information, see Create or modify a tag name.

{
  "PipelineMetaTagKey": {
    "HOST_NAME": "sourceName",
    "AGENT_TAG": "",
    "HOST_ID": "__default__",
    "CLOUD_PROVIDER": "__default__"
  }
}

View tags

1. In the Projects section, click the one you want.

   

2. To the right of the target Logstore, click the icon and select Query and Analysis.

3. The tag fields are displayed in the top row of the log page.

   • Agent-related tags

     • __tag__:sourceName: This tag represents the source hostname. Its field name is set to sourceName by using the PipelineMetaTagKey.HOST_NAME key. The default value is __hostname__.

     • __tag__:__host_id__: The tag key is set to PipelineMetaTagKey.HOST_ID. The default value __host_id__ is used.

     • __tag__:__cloud_provider__: The tag key is set to PipelineMetaTagKey.CLOUD_PROVIDER. The default value __cloud_provider__ is used.

     • __tag__:__user_defined_id__: The tag key is set to AGENT_TAG. This parameter applies only to a machine group with custom identifiers and not to a machine group with IP addresses.

   • Input plugin-related tags

     • __tag__:__inode__: The tag key is set to Tags.FileInodeTagKey. The default value __inode__ is used.

     • __tag__:__path__: The tag key is set to Tags.FilePathTagKey. The default value __path__ is used.

     • __file_offset__: The tag key is set to FileOffsetKey. The default value __file_offset__ is used.

       Important

       • If you set FileOffsetKey in the input configuration, the Auto Generate Index feature automatically creates a field index for this tag (__file_offset__). For more information about billing, see Billing. If you do not need the field index, you can delete it without removing the tag. For more information, see Update an index.

       • This tag is not displayed in the top section of the Search & Analysis page. It has unique properties and features, and its name is __file_offset__.

         On the Search & Analysis page, the __file_offset__ field appears in the Displayed Fields list on the left and in the log details area. Its value is the file read offset, such as 1465.

Modify a tag value

You can use an SPL processing plugin to modify tag values. When using SPL to process tags, follow these rules:

1. SPL uses the __tag__: prefix to identify tags. This prefix is automatically added to all tags during SPL processing, so you must include it in the field name when writing SPL statements.

2. In the SPL output, any field with the __tag__: prefix is treated as a tag. You can use this method to create new tags.

This section provides an example of adding an SPL statement when modifying a collection configuration to change the value of __tag__ :sourceName. To simplify log queries, you can change the value to user_module to distinguish logs by application name. For information about how to add an SPL statement when you create a collection configuration, see Add an SPL statement when you create a collection configuration.

1. In the Projects section, click the one you want.

   

2. On the Log Storage > Logstores tab, click > next to the target Logstore, then choose Data Collection > Logtail Configuration.

3. In the Logtail Configuration list, find the target configuration and click Manage Logtail Configuration in the Actions column.

4. Click Edit at the top of the page. In the Processor Configurations area at the bottom of the page, in Processor Configurations, select SPL for Processing Method, and then click Save.

   Set the SPL statement to the following: * | extend "__tag__:sourceName"='user_module'

5. To the right of the target Logstore, click the icon and select Query and Analysis.

   The value of the __tag__:sourceName field is user_module.

Related documents

You can also configure a standard index field as a tag. The resulting tag fields can be shown or hidden. For more information, see Tag fields.