Field operation instructions - Simple Log Service - Alibaba Cloud Documentation Center

project

This instruction retains fields that match a given pattern and renames specified fields. During execution, all field retention expressions are executed first, followed by the renaming expressions.

Important

The default time fields `__time__` and `__time_ns_part__` are reserved and cannot be renamed or overwritten. For more information, see Time field.

Syntax

| project -wildcard <field-pattern>, <output>=<field>, ...

Parameters

Parameter	Type	Required	Description
wildcard	Bool	No	Specifies whether to enable wildcard pattern matching. The default is exact field matching. Add this parameter to enable wildcard pattern matching.
field-pattern	FieldPattern	Yes	The name of the field to retain, or a combination of a field name and wildcard characters to process all matched fields.
output	Field	Yes	The new name for the field. You cannot rename multiple fields to the same target field. Important If the new field has the same name as a field in the input data, see Retaining and Overwriting Old and New Values for the policy on value assignment.
field	Field	Yes	The original name of the field to rename. If the field does not exist in the input data, the rename operation is not performed. A field cannot be renamed multiple times.

Examples

Example 1: Retain specific fields.
```
* | project level, err_msg
```
Example 2: Rename fields.
```
* | project log_level=level, err_msg
```
Example 3: Retain the field that exactly matches __tag__:*.
```
* | project "__tag__:*"
```

project-away

This instruction removes fields that match a given pattern and retains all other fields.

Important

This instruction retains the time fields __time__ and __time_ns_part__ by default. For more information, see Time fields.

Syntax

| project-away -wildcard <field-pattern>, ...

Parameters

Parameter	Type	Required	Description
wildcard	Bool	No	Specifies whether to enable wildcard pattern matching. The default is exact field matching. Add this parameter to enable wildcard pattern matching.
field-pattern	FieldPattern	Yes	The name of the field to remove, or a combination of a field name and wildcard characters to process all matched fields.

project-rename

This instruction renames specified fields and retains all other fields.

Important

The default time fields `__time__` and `__time_ns_part__` are reserved and cannot be renamed or overwritten. For more information, see Time fields.

Syntax

| project-rename <output>=<field>, ...

Parameters

Parameter

Type

Required

Description

output

Field

Yes

The new name for the field. You cannot rename multiple fields to the same target field.

Important

If the new field has the same name as a field in the input data, see Retention and Overwriting of Old and New Values for the value assignment policy.

field

Field

Yes

The original name of the field to be renamed.

If the field does not exist in the data entry, the rename operation is not performed.
A field cannot be renamed multiple times.

Example

Rename specified fields.

* | project-rename log_level=level, log_err_msg=err_msg

expand-values

This instruction expands the first layer of a JSON object in a specified field to generate multiple results.

Important

The data type of the output fields is VARCHAR. If a result field has the same name as an input field, see Value Retention and Overwriting for information about the value handling policy.
You cannot perform operations on the time fields __time__ or __time_ns_part__. For more information, see Time fields.
This instruction supports data transformation in the new version. For different SPL usage scenarios, see General Reference.

Syntax

| expand-values -path=<path> -limit=<limit> -keep <field> as <output>

Parameters

Parameter	Type	Required	Description
path	JSONPath	No	Specifies the JSON path within the field content to locate the content that needs to be expanded. The default value is empty, which means the entire content of the specified field is expanded directly.
limit	Integer	No	The maximum number of entries expanded from each raw data entry. Valid values: 0 and 1 to 10. Default value: 10. Note The cumulative size of expanded values for a single field cannot exceed 50 MB. When limit is set to 1 to 10, the number of expanded rows is bounded by this value. Note When limit is set to 0, the number of expanded rows per field is unrestricted. However, total returned data across all fields and rows is capped at approximately 50 MB. If this cap is exceeded, only partial results already processed are returned. Individual field values are not truncated. Note The value that triggers the cap is included in the result, so the actual returned data may slightly exceed 50 MB. Truncation due to the cap is silent and does not generate any error or warning.
keep	Bool	No	Specifies whether to retain the original field after expansion. By default, the original field is not retained. Add this parameter to retain the original field.
field	Field	Yes	The name of the original field to expand. The supported type is `VARCHAR`. If the specified field does not exist, the expansion operation is not performed.
output	Field	No	The name of the target field for the expansion. If not specified, the result is output to the input field by default. The expansion logic for the original content is as follows: JSON array: Expands each element of the array. JSON dictionary: Expands each key-value pair of the dictionary. Other JSON types: Returns the initial value. Invalid JSON: Returns `null`.

Examples

Example 1: Expand an array to output multiple data results.

SPL statement
```
* | expand-values y
```
Input data
```
x: 'abc'
y: '[0,1,2]'
```

Output data

# Entry 1
x: 'abc'
y: '0'

# Entry 2
x: 'abc'
y: '1'

# Entry 3
x: 'abc'
y: '2'

Example 2: Expand a dictionary to output multiple data results.

SPL statement
```
* | expand-values y
```
Input data
```
x: 'abc'
y: '{"a": 1, "b": 2}'
```

Output data

# Entry 1
x: 'abc'
y: '{"a": 1}'

# Entry 2
x: 'abc'
y: '{"b": 2}'

Example 3: Expand the content of a specified JSON path and output it to a new field.

SPL statement

* | expand-values -path='$.body' -keep content as body

Input data

content: '{"body": [0, {"a": 1, "b": 2}]}'

Output data

# Entry 1
content: '{"body": [1, 2]}'
body: '0'

# Entry 2
content: '{"body": [1, 2]}'
body: '{"a": 1, "b": 2}'