All Products
Search
Document Center

Simple Log Service:Data types

Last Updated:Feb 08, 2025

When creating indexes, you can specify the data type of a field as text, long, double, or JSON. This topic explains the various data types for fields and includes examples.

Overview of data types

Simple Log Service supports the following data types for index creation. Please choose according to your requirements:

  • Text, long, double types:

    • To query and analyze log data of the string type, configure the related fields' data type as text and enable analytics for these fields.

    • Fields with long or double data types allow range queries. However, you cannot use an asterisk (*) or a question mark (?) for fuzzy queries.

      • Set the data type to long for fields with integer values.

      • Set the data type to double for fields with floating-point number values, to enable querying.

      • If you set a field's data type to long or double but the field value is a string, querying the field will fail.

      Important

      To query logs containing invalid numbers, use the statement not key > -1000000, where -1000000 is a sufficiently small value.

  • JSON type: Set the data type to JSON for fields with JSON values when configuring indexes.

    • For fields within JSON objects, set the data type to long, double, or text based on the field value and enable analytics. Once analytics is enabled, Simple Log Service supports querying and analyzing the field.

      By selecting Automatically Index All Text Fields In JSON, all text fields within JSON objects are automatically indexed, generating index traffic. image.png

    • Simple Log Service can parse only the valid parts of partially valid JSON-formatted data.

      For instance, Simple Log Service can correctly parse the content.remote_addr, content.request.request_length, and content.request.request_method fields in the following incomplete JSON log.

      content: {
           remote_addr:"192.0.2.0"
           request: {
                   request_length:"73"
                   request_method:"GE
      Important
      • You can configure indexes for leaf nodes within JSON objects, but not for child nodes that contain leaf nodes.

      • Fields with JSON array values cannot have configured indexes, nor can the fields within a JSON array.

      • Set the data type to text for fields with Boolean values when configuring indexes.

      • The format for query and analytic statements is query statement|analytic statement. Enclose field names in double quotation marks ("") and strings in single quotation marks ('') within an analytic statement.

    • For more information

      • For detailed scenarios and FAQs on querying and analyzing JSON logs, including index configuration, querying and analyzing JSON fields with indexes, using JSON functions, and analyzing JSON arrays, see FAQs about querying and analyzing JSON logs.

      • For basic configurations and usage related to querying and analyzing JSON logs, see Querying and analyzing JSON logs.

      • If the data volume of JSON logs you want to query and analyze is small, configuring indexes for JSON leaf nodes is unnecessary. In such cases, you can use JSON functions for querying and analysis. For related scenarios, see When to use JSON functions. For a comprehensive introduction and examples of JSON functions, see JSON functions.

Procedure

  1. Create indexes in the console by following the steps outlined in Create Index.

  2. Select the appropriate index type in the index configuration.

    text, long, double types

    • Log sampletext log sample

    • Configure indextext index configuration

    • Examples of query and analytic statements

      • Query logs where the request time exceeds 60 seconds: request_time > 60.

      • Query logs with request times between 60 and 200 seconds: request_time in [60 200) or request_time >= 60 and request_time < 200.

      • Query logs with a status code of 200: status = 200.

      • Query logs for non-GET requests: not request_method : GET.

      • Query logs starting with 'cn': cn*.

      • Count the distribution of clients: * | SELECT ip_to_province(client_ip) as province, count(*) AS pv GROUP BY province ORDER BY pv.

    JSON type

    • Log sample

      Below is a sample of JSON logs. In addition to Simple Log Service's reserved fields, the logs include the class, latency, status, and info fields. The info field is a JSON object with multilayer nesting.

      JSON log sample

    • Configure indexconfigure index

      The following section describes the networking rules in EDS:

      • The values of the IP field and the data field are JSON arrays. Therefore, you cannot configure indexes for the IP field and the data field or query and analyze the fields.

      • The region field and the CreateTime field are in JSON arrays. Therefore, you cannot configure indexes for the region field and the CreateTime field or query and analyze the fields.

    • Examples of query and analytic statements

      • Query logs where the usedTime field value exceeds 60 seconds: info.usedTime > 60.

      • Query logs where the success field is true: info.success : true.

      • Query logs where the usedTime field value exceeds 60 seconds and the projectName field is not project01: info.usedTime > 60 not info.param.projectName : project01.

      • Calculate the average time to obtain project information: methodName = getProjectInfo | SELECT avg("info.usedTime") AS avg_time.