Simple Log Service:Manage LoongCollector collection tags

Limits

• This feature is supported only in LoongCollector 3.0.10 and later. If you use Logtail or an earlier version of LoongCollector, upgrade to the latest version of LoongCollector.

• This feature modifies the names and storage locations of tags. If your log consumption process depends on the modified tag fields, compatibility issues may occur.

Tag classification

A tag field is similar to a field index and contains a key and a value. For example, in __tag__:__inode__:263554, __tag__:__inode__ is the tag name (key), and 263554 is the tag value.

Tags are classified into two main types based on their source:

• Agent-related: These tags are related to the collection agent itself and do not depend on plugins. Examples include IP addresses and hostnames.

• Input plugin-related: These tags depend on input plugins. The plugins provide and enrich the logs with relevant information, such as file inodes, read offsets, pod names, namespaces, and image names.

Billing

Tag fields are not charged. However, fees are incurred if you set them as index fields. For more information about billing standards, see Billing. For more information about how to set and delete field indexes, see Create an index.

Create or modify a tag name

LoongCollector standardizes tags and provides a tag processing feature. You can configure advanced parameters to add, delete, and rename tags.

Delete a tag

To delete a tag, set its value to an empty string in the advanced parameter configuration.

In the following example, the value of AGENT_TAG is set to an empty string, so the tag is not displayed. For more information, see Create or modify a tag name.

{
  "PipelineMetaTagKey": {
    "HOST_NAME": "sourceName",
    "AGENT_TAG": "",
    "HOST_ID": "__default__",
    "CLOUD_PROVIDER": "__default__"
  }
}

View tags

1. In the Projects section, click the one you want.

   

2. To the right of the target Logstore, click the icon and select Search & Analysis to view the logs of the current Logstore.

   

3. As shown in the following figure, the tag fields are displayed in the top row of the log page.

   

   • Agent-related tags

     • __tag__:sourceName: The tag key is set to PipelineMetaTagKey.HOST_NAME. The default value is __tag__:__hostname__. In this example, the value is set to __tag__:sourceName, which represents the source hostname.

     • __tag__:__host_id__: The tag key is set to PipelineMetaTagKey.HOST_ID. The default value __host_id__ is used for this parameter.

     • __tag__:__cloud_provider__: The tag key is set to PipelineMetaTagKey.CLOUD_PROVIDER. The default value __cloud_provider__ is used for this parameter.

     • __tag__:__user_defined_id__: The tag key is set to AGENT_TAG. This parameter applies only to machine groups with custom identifiers and not to machine groups with IP addresses.

   • Input plugin-related tags

     • __tag__:__inode__: The tag key is set to Tags.FileInodeTagKey. The default value __inode__ is used for this parameter.

     • __tag__:__path__: The tag key is set to Tags.FilePathTagKey. The default value __path__ is used for this parameter.

     • __file_offset__: The tag key is set to FileOffsetKey. The default value __file_offset__ is used for this parameter.

       Important

       • If you set FileOffsetKey in the input configuration, the Auto Generate Index feature of the collection configuration sets this tag field (__file_offset__) as an index field. For more information about billing, see Billing. If you do not need to create an index for this field, you can delete the index but keep the tag field. For more information, see Update an index.

       • This tag is not displayed at the top of the query analysis page. It has unique properties and features, and its name is __file_offset__. The following figure shows an example:

         

Modify a tag value

You can use the Structured Process Language (SPL) processing plugin to manage tag fields and modify tag values. When you use SPL to process tag fields, follow these rules:

1. SPL uses the __tag__: prefix to identify tags and automatically adds this prefix to all tag fields during processing. Therefore, when you write an SPL statement, make sure that the field name includes this prefix to correctly reference tag-related data.

2. In the SPL output, all fields with the tag prefix are identified as tags. You can use this method to add tags.

This topic describes how to add an SPL statement when you modify a collection configuration, using __tag__ :sourceName as an example. To query logs more easily, you can change the value to user_module in the log collection configuration to distinguish different logs by application name. For more information about how to add an SPL statement when you create a collection configuration, see Add an SPL statement when you create a collection configuration.

1. In the Projects section, click the one you want.

   

2. On the Log Storage > Logstores tab, click the > icon next to the logstore you want, and then choose Data Collection > Logtail Configurations.

   

3. In the Logtail Configuration list, find the required Logtail configuration and click Manage Logtail Configuration in the Actions column.

4. Click Edit at the top of the page. In the Processing Configurations section at the bottom of the page, set Processing Method in Processing Configurations to SPL, and then click Save.

   Set the SPL statement to the following: * | extend "__tag__:sourceName"='user_module'

5. To the right of the target Logstore, click the icon and select Search & Analysis to view the logs of the current Logstore.

   The value of the __tag__:sourceName field is user_module.

   

References

You can set a normal index field as a tag field. Tag fields can be shown or hidden. For more information, see Tag fields.