Note No topic can be set for a syslog.

Topic generation modes

You can set a topic when using Logtail to collect logs or using APIs/SDKs to upload data. Currently, the following topic generation modes are supported in the console: Null - Do not generate topic, Machine Group Topic Attributes, and File Path Regular.
  • Null - Do not generate topic

    The default log topic generation mode is Null - Do not generate topic when you configure Logtail to collect text logs in the console, that is, the topic is a null string, and you can query logs without entering a topic.

  • Machine Group Topic Attributes

    Machine Group Topic Attributes mode is used to clearly differentiate log data generated in different servers. If the log data of different servers are stored in the same file path and file name, you can divide the machines into different machine groups to distinguish the log data of different servers by topic. To do this, set different topic attributes for different machine groups when creating machine groups, and set the Topic Generation Mode to Machine Group Topic Attributes. Apply the previously created Logtail configuration to those machine groups to complete the configuration.

    With this mode selected, Logtail uploads the topic attribute of the machine group where the current machine belongs as the topic name to Log Service when reporting data. When querying logs, you must specify a topic, that is, you must specify the topic attribute of the target machine group as the query condition.

  • File Path Regular

    This mode is used to differentiate log data generated by users and instances. If service logs are stored under different directories based on users or instances but their subdirectories and log file names are the same, Log Service cannot clearly differentiate which user or instance generates the logs when collecting log files. In this case, you can set the Topic Generation Mode to File Path Regular, enter a regular expression of file paths, and set the topic as the instance name.

    With this mode selected, Logtail uploads the instance name as the topic name to Log Service when reporting data. Different topics are generated according to your directory structure and configuration. You must specify the topic name as the instance name when querying logs.

    For example, the following directory setting stores logs in different directories according to the users to which the logs belong:
      | - /userA/serviceA
        | - service.log
      | - /userB/serviceA
        | - service.log
      | - /userC/serviceA
        | - service.log

    For files that are named service.log and stored in /logs in th server, the users or instances from which the logs of these files were generated cannot be distinguished. You can set Topic Generation Mode as File Path Regular, and enter \/(.*)\/serviceA\/.* to extract the instance names. This method can generate an unique topic for logs in each directory: userA, userB, and userC. Then, you can query a log by specifying its topic.

    Note You must escape / in the regular expression for a file directory.

Set log topic

  1. Configure Logtail in the console by following the Collect text logs.

    To set the topic generation mode to Machine Group Topic Attributes, configure the Machine Group Topic when creating/modifying a machine group.

  2. Expand Advanced Options in the data import wizard and select a topic generation mode from the Topic Generation Mode drop-down list.
Figure 1. Set log topic

Modify log topic

To change the log topic generation mode, modify the Topic Generation Mode option directly in the data import wizard.

Note The modified configuration only applies to the data collected after the modification takes effect.