SLS time-processing plugins - parse and standardize log timestamps - Simple Log Service

Example of plugin effect

Comparison of raw log data in SLS with and without the time parsing plugin (native).

Raw log	Without plugin	Time parsing plugin (native)
Second-precision timestamp: `{ "level":"INFO", "timestamp":"2025-09-29T09:56:01+0800", "cluster":"yilu-cluster-0728", "message":"User logged in successfully", "userId":"user-123" }`	Content: "{"level":"INFO","timestamp":"2025-09-29T09:56:01+0800","cluster":"yilu-cluster-0728","message":"User logged in successfully","userId":"user-123"}"
Millisecond-precision timestamp: `{ "time": "2026-01-05T11:58:40,647Z", "filename": "out_data.py", "levelname": "INFO", "threadName": "MainThread" }`	Content: "{"time":"2026-01-05T11:58:40,647Z", "filename":"out_data.py","levelname": "INFO", "threadName":"MainThread"}"
Nanosecond-precision timestamp: `{ "time": "2026-01-05T11:40:22,298837465Z07:00", "filename": "out_data.py", "levelname": "INFO", "threadName": "MainThread" }`	Content: "{"time": "2026-01-05T11:40:22,298837465Z07:00","filename":"out_data.py","levelname":"INFO","threadName": "MainThread"}"

Time processing plugins

SLS provides the following time-processing plugins.

Name	Type	Description
time parsing	native	Parses and standardizes the time field in logs.
extract log time	extension	Parses the raw time field and sets it as the log timestamp.

Entry point

To use a Logtail plugin for log processing, add it when you create or modify a Logtail configuration. For more information, see Overview.

Differences between native and extension plugins

Native plugins: Implemented in C++ for higher performance.

Extension plugins: Implemented in Go for a richer, more flexible ecosystem. Use extension plugins when native plugins cannot handle your log format.

Performance limits of extension plugins
- Extension plugins consume more resources (primarily CPU). Adjust LoongCollector parameters in configuration management if needed.
- If raw data exceeds 5 MB/s, avoid complex plugin combinations. Use a simple extension plugin and offload further processing to Data Transformation.
Log collection limits
- Extension plugins process text logs in line mode. This means that file-level metadata, such as __tag__:__path__ and __topic__, is stored in each log entry.
- Adding an extension plugin affects tag-related features:
  - Context query and LiveTail become unavailable unless you add an aggregators configuration.
  - The __topic__ field is renamed to __log_topic__. When you add the aggregators configuration, the log contains both the __topic__ and __log_topic__ fields. If you do not need the __log_topic__ field, you can use the drop field plugin to delete it.
  - Fields such as __tag__:__path__ lose their native field index. You must create an index for them.

Time parsing plugin (native)

Parses the time field in a log and assigns the result to the __time__ field.

Configuration

Parameter	Description
Original Field	The field containing the log content to parse. Default: content. Note When using the regex parsing plugin, set the source field to time and include time as an extracted field in your regex configuration.
Time Format	The format of the timestamp in the log. For example, if the timestamp is 10/Sep/2023:12:36:49, the corresponding time format is %d/%b/%Y:%H:%M:%S.
Time Zone	The time zone of the timestamp in the log. If left empty, the plugin uses the time zone of the machine where Logtail is running.

By default, Simple Log Service processes timestamps with second precision. You only need to configure the time format down to the second and can omit smaller units like milliseconds or microseconds. To retain millisecond, microsecond, or nanosecond precision, enable nanosecond precision as shown below. Log collection that supports nanosecond timestamps.

Raw log

Time parsing plugin configuration

Time format

{
  "time": "2026-01-05T11:40:22,298837465Z07:00",
  "filename": "out_data.py",
  "levelname": "INFO",
  "threadName": "MainThread"
}

%Y-%m-%dT%H:%M:%S,%f

To enable nanosecond precision:

On the Logtail configuration page, navigate to Global Configurations > Other Global Configurations, enable Advanced Parameters, and enter:

{
  "EnableTimestampNanosecond": true
}

Also enable EnableTimestampNanosecond when using extension plugins to parse timestamps with nanosecond or millisecond precision.

Log time extractor plugin

Use processor_gotime or processor_strptime to parse the time field in raw logs.

Note

To preserve sub-second precision, see Log collection that supports nanosecond timestamps.

Go time format (`processor_gotime`)

The processor_gotime plugin parses time fields using Go time format and can set the result as the log time in SLS.

Important

Logtail 0.16.28 and later support the processor_gotime plugin.
Form-based configuration: Available when you collect text logs and container standard output.
JSON configuration: Not available when you collect text logs.

Form configuration

Parameters

Set Processor Type to Extract Log Time (Go Time Format).

Parameters:

Parameter	Description
Original Time Field	The name of the field that contains the time to parse.
Original Time Format	The Go time format string for the source field.
Original Time Zone	The time zone of the source time. If you select System Time Zone, Logtail uses the time zone of its host or container.
New Time Field	The name of the field to store the parsed and formatted time. You cannot set this parameter to `__time__`.
New Time Format	The Go time format string for the destination field.
Custom New Time Zone	The time zone for the destination time field. If you select System Time Zone, Logtail uses the time zone of its host or container.
Advanced parameters > Use as Log Time	Specifies whether to use the parsed time as the log time.
Advanced parameters > Retain Original Field	Specifies whether to retain the source field in the processed log.
Advanced parameters > Report error when source field is missing	Specifies whether to report an error if the source field is missing from the raw log.
Advanced parameters > Report error on extraction failure	Specifies whether to report an error if time extraction fails.

Example
Parses the s_key field in 2006-01-02 15:04:05 format (UTC+8), converts it to 2006/01/02 15:04:05 (UTC+9), stores the result in d_key, and sets it as the log time.
- Raw log
```
"s_key":"2022-07-05 19:28:01"
```
- Logtail plugin configuration
- Result
```
"s_key":"2022-07-05 19:28:01"
"d_key":"2022/07/05 20:28:01"
```

JSON configuration

Parameters

Set type to processor_gotime. Parameters in the detail object:

Parameter	Type	Required	Description
SourceKey	String	Yes	The name of the field that contains the time to parse.
SourceFormat	String	Yes	The Go time format string for the source field.
SourceLocation	Int	Yes	The UTC offset in hours for the source time zone. For example, use 8 for UTC+8. If you leave this parameter empty, Logtail uses the time zone of its host or container.
DestKey	String	Yes	The name of the field to store the parsed and formatted time. You cannot set this parameter to `__time__`.
DestFormat	String	Yes	The Go time format string for the destination field.
DestLocation	Int	No	The UTC offset in hours for the destination time zone. If you leave this parameter empty, Logtail uses the time zone of its host or container.
SetTime	Boolean	No	Specifies whether to use the parsed time as the log time. `true` (default): Uses the parsed time as the log time. `false`: Does not use the parsed time as the log time.
KeepSource	Boolean	No	Specifies whether to keep the source field in the processed log. `true` (default): Keeps the source field. `false`: Discards the source field.
NoKeyError	Boolean	No	Specifies whether to report an error if the source field is missing from the raw log. `true` (default): Reports an error. `false`: Does not report an error.
AlarmIfFail	Boolean	No	Specifies whether to report an error if time extraction fails. `true` (default): Reports an error. `false`: Does not report an error.

Example

Parses the s_key field in 2006-01-02 15:04:05 format (UTC+8), converts it to 2006/01/02 15:04:05 (UTC+9), stores the result in d_key, and sets it as the log time.

Raw log
```
"s_key":"2019-07-05 19:28:01"
```

Logtail plugin configuration

{
  "processors":[
    {
      "type":"processor_gotime",
      "detail": {
        "SourceKey": "s_key",
        "SourceFormat":"2006-01-02 15:04:05",
        "SourceLocation":8,
        "DestKey":"d_key",
        "DestFormat":"2006/01/02 15:04:05",
        "DestLocation":9,
        "SetTime": true,
        "KeepSource": true,
        "NoKeyError": true,
        "AlarmIfFail": true
      }
    }
  ]
}

Result

"s_key":"2019-07-05 19:28:01"
"d_key":"2019/07/05 20:28:01"

Strptime time format (`processor_strptime`)

The processor_strptime plugin parses time fields using the Linux strptime format and can set the result as the log time.

Important

The processor_strptime plugin is available in Logtail 0.16.28 and later.

Form

Parameters

Set processor type to Extract Log Time (strptime Time Format).

Parameter	Description
Original Field	The name of the source field.
Original Time Format	The format of the source time.
Retain Original Field	Retains the source field in the processed log.
Report Extraction Failure Error	Raises an alarm if time extraction fails.
Use Time Offset	Enables setting a custom time offset in seconds.
Time Offset (Unit: Seconds)	The time offset in seconds. For example, 28800 indicates UTC+8, and -3600 indicates UTC-1.

Configuration example
Parses a time in %Y/%m/%d %H:%M:%S format from the log_time field and sets it as the log time. Assumes the host time zone is UTC+8.
- Raw log
```
"log_time":"2016/01/02 12:59:59"
```
- Logtail plugin configuration
- Processing result
```
"log_time":"2016/01/02 12:59:59"
Log.Time = 1451710799
```

Common time expressions

Note

The processor_strptime plugin supports the %f format specifier, which represents the fractional part of a second with precision up to nanoseconds.

Example	Time expression
2016/01/02 12:59:59	%Y/%m/%d %H:%M:%S
2016/01/02 12:59:59.1	%Y/%m/%d %H:%M:%S.%f
2016/01/02 12:59:59.987654321 +0700 (UTC)	%Y/%m/%d %H:%M:%S.%f %z (%Z)
2016/Jan/02 12:59:59,123456	%Y/%b/%d %H:%M:%S,%f
2019-07-15T04:16:47.123Z	%Y-%m-%dT%H:%M:%S.%f

JSON

Parameters

Set type to processor_strptime. Parameters in the detail object:

Parameter	Type	Required	Description
SourceKey	String	Yes	The name of the source field.
Format	String	Yes	The format of the source time.
AdjustUTCOffset	Boolean	No	Specifies whether to adjust the time zone. true: Adjusts the time zone. false (default): Does not adjust the time zone.
UTCOffset	Int	No	The time zone offset in seconds. This setting applies only when AdjustUTCOffset is true.
AlarmIfFail	Boolean	No	Specifies whether to raise an alarm if the plugin fails to extract the log time. true (default): Raises an alarm. false: Does not raise an alarm.
KeepSource	Boolean	No	Specifies whether to retain the source field in the processed log. true (default): Retains the source field. false: Discards the source field.

Examples

Parses a time in %Y/%m/%d %H:%M:%S format from the log_time field and sets it as the log time. Logtail uses the host machine time zone.

Example 1: Assume the time zone is UTC+8.

Raw log
```
"log_time":"2016/01/02 12:59:59"
```

Logtail plugin configuration

{
  "processors":[
    {
      "type":"processor_strptime",
      "detail": {
        "SourceKey": "log_time",
        "Format": "%Y/%m/%d %H:%M:%S"
      }
    }
  ]
}

Processing result

"log_time":"2016/01/02 12:59:59"
Log.Time = 1451710799

Example 2: Assume the time zone is UTC+7.

Raw log
```
"log_time":"2016/01/02 12:59:59"
```

Logtail plugin configuration

{
  "processors":[
    {
      "type":"processor_strptime",
      "detail": {
        "SourceKey": "log_time",
        "Format": "%Y/%m/%d %H:%M:%S",
        "AdjustUTCOffset": true,
        "UTCOffset": 25200
      }
    }
  ]
}

Processing result

"log_time":"2016/01/02 12:59:59"
Log.Time = 1451714399

Example 3: Assume the time zone is UTC+7.

Raw log
```
"log_time":"2016/01/02 12:59:59.123"
```

Logtail plugin configuration

{
  "processors":[
    {
      "type":"processor_strptime",
      "detail": {
        "SourceKey": "log_time",
        "Format": "%Y/%m/%d %H:%M:%S.%f"
      }
    }
  ]
}

Processing result

"log_time":"2016/01/02 12:59:59.123"
Log.Time = 1451714399

Common time expressions

Note

The processor_strptime plugin supports the %f format specifier, which represents the fractional part of a second with precision up to nanoseconds.

Example	Time expression
2016/01/02 12:59:59	%Y/%m/%d %H:%M:%S
2016/01/02 12:59:59.1	%Y/%m/%d %H:%M:%S.%f
2016/01/02 12:59:59.987654321 +0700 (UTC)	%Y/%m/%d %H:%M:%S.%f %z (%Z)
2016/Jan/02 12:59:59,123456	%Y/%b/%d %H:%M:%S,%f
2019-07-15T04:16:47.123Z	%Y-%m-%dT%H:%M:%S.%f

Log time formats

For processor_gotime time formats, see https://pkg.go.dev/time#pkg-constants. The processor_parse_timestamp_native native plugin and processor_strptime extension plugin support the formats listed below.

Note

On a Linux server, Logtail supports all time formats provided by the strftime function.

Format	Description	Example
%a	Abbreviated weekday name.	Fri
%A	Full weekday name.	Friday
%b	Abbreviated month name.	Jan
%B	Full month name.	January
%d	Day of the month as a zero-padded decimal number (01-31).	07, 31
%f	Fractional seconds, such as milliseconds, microseconds, or nanoseconds.	123
%h	Abbreviated month name. Same as %b.	Jan
%H	Hour (24-hour clock) as a zero-padded decimal number (00-23).	22
%I	Hour (12-hour clock) as a zero-padded decimal number (01-12).	11
%m	Month as a zero-padded decimal number (01-12).	08
%M	Minute as a zero-padded decimal number (00-59).	59
%n	A line feed.	line feed
%p	Either AM or PM.	AM, PM
%r	Time in 12-hour format. Same as %I:%M:%S %p.	11:59:59 AM
%R	Hour and minute in 24-hour format. Same as %H:%M.	23:59
%S	Second as a zero-padded decimal number (00-59).	59
%t	A tab character.	None
%y	Year without century as a zero-padded decimal number (00-99).	04, 98
%Y	Year with century as a decimal number.	2004, 1998
%C	Century as a decimal number (00-99).	16
%e	Day of the month as a space-padded decimal number (1-31).	7, 31
%j	Day of the year as a zero-padded decimal number (001-366).	365
%u	Weekday as a decimal number, where Monday is 1 (1-7).	2
%U	Week number of the year, with Sunday as the first day of the week (00-53).	23
%V	ISO 8601 week number of the year, with Monday as the first day of the week (01-53). Week 1 is the first week of the year that contains at least four days.	24
%w	Weekday as a decimal number, where Sunday is 0 (0-6).	5
%W	Week number of the year, with Monday as the first day of the week (00-53).	23
%c	Locale-specific date and time representation.	Tue Nov 20 14:12:58 2020
%x	Locale-specific date representation.	Tue Nov 20 2020
%X	Locale-specific time representation.	11:59:59
%s	Unix timestamp.	1476187251

Examples

Common time standards and their corresponding expressions:

Example	Expression	Standard
2017-12-11 15:05:07	%Y-%m-%d %H:%M:%S	Custom
[2017-12-11 15:05:07.012]	[%Y-%m-%d %H:%M:%S.%f]	Custom
2017-12-11 15:05:07.123	%Y-%m-%d %H:%M:%S.%f	Custom
02 Jan 06 15:04 MST	%d %b %y %H:%M %Z	RFC822
02 Jan 06 15:04 -0700	%d %b %y %H:%M %z	RFC822Z
Monday, 02-Jan-06 15:04:05 MST	%A, %d-%b-%y %H:%M:%S %Z	RFC850
Mon, 02 Jan 2006 15:04:05 MST	%a, %d %b %Y %H:%M:%S %Z	RFC1123
2006-01-02T15:04:05Z07:00	%Y-%m-%dT%H:%M:%S%z	RFC3339
2006-01-02T15:04:05.999999999Z07:00	%Y-%m-%dT%H:%M:%S.%f%z	RFC3339Nano
1637843406	%s	Custom
1637843406123	%s	Custom (Simple Log Service truncates timestamps to second-level precision)

Simple Log Service:Time-processing plugins

Example of plugin effect

Time processing plugins

Entry point

Differences between native and extension plugins

Time parsing plugin (native)

Configuration

Log time extractor plugin

Go time format (`processor_gotime`)

Form configuration

JSON configuration

Strptime time format (`processor_strptime`)

Form

JSON

Log time formats

Examples

Related documents

Example of plugin effect

Time processing plugins

Entry point

Differences between native and extension plugins

Time parsing plugin (native)

Configuration

Log time extractor plugin

Go time format (processor_gotime)

Form configuration

JSON configuration

Strptime time format (processor_strptime)

Form

JSON

Log time formats

Examples

Related documents

Go time format (`processor_gotime`)

Strptime time format (`processor_strptime`)