This topic describes how to enable and configure the index feature for a Logstore.

Background information

Before querying the log data collected in a Logstore, you must enable and configure the index feature for the Logstore. You can configure indexes based on your business requirements.
Note After the search and analytics feature is enabled, data is indexed on the backend server. Index traffic is incurred and index storage space is required.
When collecting a log entry, Log Service adds the relevant information (such as the source and time fields) to the log entry as key-value pairs. These fields are reserved in Log Service. When you enable and configure the index feature for a Logstore, the index and analytics features are enabled for the fields in the Logstore.
Table 1. Reserved fields in Log Service
Field Description
__topic__ The topic of a log entry. If you specify a log topic, Log Service automatically adds the topic field to your log entries with the key set to __topic__ and the value set to the topic content that you specified.
__source__ The source of a log entry. This field indicates the source device that generates the log entry.
__time__ The time that is specified when a log entry is written to the Logstore by the SDK.
Note The delimiters configured for the values of the __topic__ and __source__ fields are null. This means that the keywords used to query the two fields must exactly match the field values.

Procedure

  1. Log on to the Log Service console, and then click the target project name.
  2. Click the Logstore management icon icon next to the name of the Logstore, and then select Search & Analysis.
  3. In the upper-right corner of the page that appears, click Enable.
    Enable indexing
    Note If you have created an index for the Logstore, you can choose Index Attributes > Modify to modify the index.
  4. Configure indexes.
    Note If a field has the full text index and field index configured, the field index takes effect when you query the field.
    Table 2. Index types
    Index type Description
    Full text index All fields are indexed in the text format. You can query keys and values in the text format. To query fields of the long type, you must specify the field name. For fields of other types, you do not need to specify the field name in queries.
    Field index When you configure a field index for a field, you must specify the field name during queries. If a field index is configured for a field, the full text index configured for the field does not take effect.
    You can configure a field index for the following data types:
    • Configure a full text index.
      By default, a full text index allows you to query the values of all fields during a log query.
      Table 3. Full text index
      Configuration item Description Example
      Full Text Index Specifies whether to query the full content of a log. By default, a full text index allows you to query the values of all fields during a log query. A log entry is retrieved if any field value in the log matches the keyword. -
      Case Sensitive Specifies whether to match field values that contain same letters as the keyword but in different cases during queries.
      • If you disable this switch, field values that contain same letters as the keyword but in different cases are matched during queries. For example, if a log contains internalError, the log can be retrieved by using both INTERNALERROR and internalerror as the keyword.
      • If you enable this switch, field values that contain same letters as the keyword but in different cases cannot be matched during queries. For example, if a log contains internalError, the log can be retrieved by using only internalError as the keyword.
      -
      Delimiter Specifies whether to use one-character delimiters to delimit the content of a log entry so that you can use multiple keywords to match the same log entry.

      For example, if the content of a log entry is a,b;c;D-F, you can specify commas (,), semicolons (;), and hyphens (-) as delimiters to delimit the log content. Then you can use the five letters a, b, c, D, and F as keywords to match the log entry.

      , '";=()[]{}? @&<>/:\n\t
    • Configure a field index.

      You can configure a field index for specified fields. Field indexes allow you to narrow down the scope and query values of specified fields in a log.

      Table 4. Field index
      Configuration item Description Example
      Key Name The name of a field in the log.
      Note
      • If you want to configure an index for a Tags field, such as an Internet IP address or Unix timestamp, you must set the Key Name item in the format of __tag__:key, for example, __tag__:__receive_time__. For more information, see Log.
      • A Tags field does not support numeric indexes. You must set the Type of all Tags fields to text.
      _address_
      Type The data type of a field value in the log. Valid values:
      • text: The value of the field is in the text format.
      • long: The value of the field is an integer. You must use a numeric value range to query field values of this data type.
      • double: The value of the field is a floating-point number. You must use a numeric value range to query field values of this data type.
      • json: The value of the field is in the JSON format.
      Note The Case Sensitive and Delimiter items are not available when you configure a field index for a field whose value data type is long or double.
      -
      Alias The alias of a column name.

      An alias is used only for SQL analytics. You must use the original name for underlying storage and queries. For more information, see Column alias.

      address
      Case Sensitive Specifies whether to match field values that contain same letters as the keyword but in different cases during queries.
      • If you disable this switch, field values that contain same letters as the keyword but in different cases are matched during queries. For example, if a log contains internalError, the log can be retrieved by using both INTERNALERROR and internalerror as the keyword.
      • If you enable this switch, field values that contain same letters as the keyword but in different cases cannot be matched during queries. For example, if a log contains internalError, the log can be retrieved by using only internalError as the keyword.
      -
      Delimiter Specifies whether to use one-character delimiters to delimit the content of a log entry so that you can use multiple keywords to match the same log entry.

      For example, if the content of a log is a,b;c;D-F, you can specify commas (,), semicolons (;), and hyphens (-) as delimiters to delimit the log content. Then you can use the five letters a, b, c, D, and F as keywords to match the log entry.

      , '";=()[]{}? @&<>/:\n\t
      Enable Analytics Specifies whether to enable the analytics feature. This feature is enabled by default.

      When this feature is enabled, you can use analytic statements to analyze query results.

      -
  5. Click OK to complete the configurations.
    Note
    • The index configurations take effect within 1 minute.
    • A modified or newly enabled index applies only to log data that is written after the index configurations.