Log - Simple Log Service - Alibaba Cloud Documentation Center

Logs are records of changes that occur in a system when the system is running. The records contain information about the operations that are performed on specific objects and the results of the operations. The records are ordered by time.

Formats

Log data is stored in different formats such as log files, events, binary logs, and metrics. Simple Log Service uses a semi-structured data model to define logs. A log consists of the following data domains: log topic, log time, log content, log source, and log tag. Simple Log Service has different format requirements for each data domain. The following table describes the data domains and format requirements.

Data domain	Description	Format
Log topic	The reserved field `__topic__` in Simple Log Service specifies the topic of a log. You can specify different log topics. For example, you can specify access_log and operation_log for website logs based on the types of the logs. For more information, see Topic.	The field value is a string of up to 128 bytes. The field value can be an empty string. If the field value is an empty string, no log topic is specified.
Log time	The reserved field `__time__` in Simple Log Service specifies the time of a log. For more information, see Reserved fields.	The field value is a UNIX timestamp.
Log content	The content of a log consists of one or more items. Each item is a `key-value` pair. If you use Logtail to collect single-line or multi-line logs in simple mode, Logtail does not parse the content of the logs. Each raw log is uploaded to the content field.	A `key-value` pair must comply with the following requirements: The key specifies the name of the field. The key must be a UTF-8 encoded string. The key can contain letters, digits, and underscores (_). The key cannot start with a digit. The string is 1 to 128 bytes in length. You cannot use the following fields: __time__ __source__ __topic__ __partition_time__ _extract_others_ __extract_others__ The value specifies the value of the field. The value can be a string of up to 1 MB.
Log source	The reserved field `__source__` in Simple Log Service specifies the source of a log. For example, the field value can be the IP address of the server for which a log is generated.	The field value is a string of up to 128 bytes.
Log tag	The tags of a log. Valid values: Custom tags: the tags that you add when you call the PutLogs operation to write logs to a specified Logstore. System tags: the tags that are added by Simple Log Service. The tags include __client_ip__ and __receive_time__.	A tag field value is in the dictionary format. The keys and the values in a dictionary are strings. A tag field is prefixed with __tag__: in logs.

Examples

In the following example, a website access log is used as the raw log. The following example shows the mapping between the raw log and the new log that is collected to Simple Log Service based on a specified data model.

Raw log

127.0.0.1 - - [01/Mar/2021:12:36:49  0800] "GET /index.html HTTP/1.1" 200 612 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36

New log obtained after the raw log is collected to Simple Log Service in simple mode
The raw log is uploaded to the content field.
New log obtained after the raw log is collected to Simple Log Service in full regex mode
Logtail splits the raw log into multiple key-value pairs based on the regular expression that you specify.