Simple Log Service:SPL instructions and functions

Parameter type

Description

Bool

A Boolean parameter that acts as a switch in SPL.

Char

An ASCII character parameter, enclosed in single quotation marks (''). Examples: 'a' for the character a, '\t' for the tab character, '\11' for the ASCII character of octal 11, and '\x09' for the ASCII character of hexadecimal 09.

Integer

An integer parameter.

String

A string parameter, enclosed in single quotation marks (''). Example: 'this is a string'.

RegExp

An RE2 regular expression parameter, enclosed in single quotation marks (''). Example: '([\d.]+)'.

RE2 syntax details: Syntax.

JSONPath

A JSON path parameter, enclosed in single quotation marks (''). Example: '$.body.values[0]'.

JSONPath syntax details: JsonPath.

Field

A field name parameter. Example: | project level, content.

Enclose field names that contain special characters (other than letters, digits, and underscores) in double quotation marks (""). Example: | project "a:b:c".

FieldPattern

A field name or wildcard pattern. The wildcard (*) matches zero or more characters. Enclose patterns in double quotation marks (""). Example: | project "__tag__:*".

SPLExp

An SPL expression parameter.

SQLExp

An SQL expression parameter.

Instruction category

Instruction name

Description

Data transformation

Ingest Processor

Consumer Processor

Scan mode (SCAN)

Logtail collection

Control instruction

.let

Defines a named dataset. SPL dataset details are covered in General reference.

Yes

×

×

√

×

Field operation instruction

project

Retains fields matching a specified pattern and renames specified fields. Field retention expressions are processed before renaming expressions.

Supported

Supported

Supported

Supported

Supported

project-away

Removes fields that match a specified pattern and retains all other fields.

Supported

Supported

Supported

Supported

✓

project-rename

Renames specified fields and retains all other fields.

Supported

Supported

Supported

Supported

Supported

expand-values

Expands the top-level JSON object of a specified field into multiple results.

Supported

×

Supported

Unsupported

×

SQL calculation instruction for structured data

extend

Creates new fields from SQL expression results. Supported SQL functions are listed in List of SQL functions supported by SPL.

Supported

Supported

Supported

Supported

Supported

where

Filters data by an SQL expression and retains matching entries. Supported SQL functions are listed in List of SQL functions supported by SPL.

Supported

✓

Supported

Supported

Supported

Semi-structured data extraction instruction

parse-regexp

Extracts information matching regular expression groups from a specified field.

√

√

Supported

✓

Supported

parse-csv

Extracts information in CSV format from a specified field.

√

Supported

Supported

√

Supported

parse-json

Extracts top-level JSON fields from a specified field.

Supported

Supported

√

√

Supported

parse-kv

Extracts key-value pair information from a specified field.

Supported

Supported

Supported

Supported

Supported

New data transformation instruction

pack-fields

Packs log fields into a new field through JSON serialization. Suitable for structured transmission scenarios such as building API request bodies.

✓

×

✓

Supported

×

log-to-metric

Converts log format to time series storage format.

Supported

×

×

×

×

metric-to-metric

Processes existing metric data, such as adding, modifying, or removing tags.

Supported

×

×

×

×

Aggregation instruction

stats

Performs statistical analysis on log fields, similar to SQL aggregate functions such as COUNT, SUM, and AVG. Supports grouping and aggregation on specific fields.

×

×

Unsupported

Supported

×

sort

Sorts query results by field values or statistical results in ascending (asc) or descending (desc) order.

×

×

×

Supported

×

limit

Limits the number of log rows returned in query results. Use limit to control data volume and avoid performance issues from large result sets.

×

×

×

Supported

×

Category

Description

Common functions

Aggregate functions

Computes a single result from a set of input values.

• avg function

• count function

• max function

• min function

• sum function

String functions

Finds, replaces, extracts, concatenates, and formats text data.

• concat function

• length function

• position function

• replace function

• split function

Date and time functions

Converts, truncates, and aggregates date and time values.

• current_date function

• date function

• day function

• date_trunc function

• time_series function

JSON functions

Extracts, transforms, and queries data from JSON objects and arrays.

• json_array_contains function

• json_array_get function

• json_array_length function

• json_extract function

• json_extract_scalar function

Regular expression functions

Matches, extracts, and replaces text using regular expressions.

• regexp_extract_all function

• regexp_extract function

• regexp_extract_bool function

• regexp_replace function

• regexp_split function

Year-on-year and month-on-month functions

Compares time-series data across different time periods.

• compare function

• ts_compare function

Array functions and operators

Accesses, filters, and transforms array elements.

• Subscript operator

• array_distinct function

• array_intersect function

• array_join function

• reverse function

map functions and operators

Operates on key-value pairs (maps).

• cardinality function

• element_at function

• histogram function

• histogram_u function

• map function

Mathematical functions

Rounds, generates random numbers, and performs trigonometric calculations.

• abs function

• ceil function

• log function

• mod function

• random function

Mathematical and statistical functions

Analyzes data distributions with correlation, regression, and probability functions.

• corr function

• covar_pop function

• regr_intercept function

• beta_cdf function

• inverse_chi_squared_cdf function

Type conversion functions

Converts values from one data type to another.

• cast function

• try_cast function

• typeof function

Window functions

Computes aggregations or rankings over a sliding window of rows.

• Aggregate functions

• dense_rank function

• lag function

IP functions

Parses and processes IP addresses.

• ip_to_city function

• ip_prefix function

• ipv6_to_city function

URL functions

Extracts components from URLs.

• url_encode function

• url_decode function

• url_extract_fragment function

• url_extract_path function

• url_extract_query function

Estimation functions

Approximates distinct counts, percentiles, and histograms for large datasets.

• approx_distinct function

• approx_percentile function

• numeric_histogram function

• numeric_histogram_u function

Binary functions

Operates on binary data.

• from_base64 function

• from_big_endian_64 function

• from_hex function

• to_hex function

• sha256 function

Bitwise operation functions

Performs bitwise operations on integers.

• bit_count function

• bitwise_and function

• bitwise_not function

• bitwise_or function

• bitwise_xor function

Spatial geometry functions

Operates on spatial geometry objects.

• ST_AsText function

• ST_GeometryFromText function

• ST_Boundary function

• ST_Buffer function

• ST_Contains function

Geographic functions

Performs geospatial analysis and calculations.

• geohash function

Color functions

Creates and manipulates color representations.

• bar function

• color function

• render function

• rgb function

HyperLogLog functions

Estimates cardinality on large datasets with minimal memory.

• approx_set function

• cardinality function

• empty_approx_set function

• merge function

Comparison operators

Compares values across types (double, bigint, varchar, timestamp, date) and returns a boolean.

• Basic operators

• ALL operator

• ANY operator

• BETWEEN operator

• DISTINCT operator

Logical operators

Combines multiple boolean expressions to control logical flow.

• AND operator

• OR operator

• NOT operator

Unit conversion functions

Converts data sizes or time intervals between different units.

• convert_data_size function

• format_duration function

Window funnel functions

Tracks sequential user actions for conversion funnel analysis.

• window_funnel function

lambda expressions

Defines inline functions for higher-order array operations in SQL or SPL.

• filter function

• reduce function

• transform function

Conditional expressions

Returns different values based on a specified condition.

• CASE WHEN expression

• IF expression

• COALESCE expression

• NULLIF expression

• TRY expression