Updates a Logtail pipeline configuration.
Try it now
Test
RAM authorization
|
Action |
Access level |
Resource type |
Condition key |
Dependent action |
|
log:UpdateLogtailPipelineConfig |
update |
*All Resource
|
|
None |
Request syntax
PUT /pipelineconfigs/{configName} HTTP/1.1
Path Parameters
|
Parameter |
Type |
Required |
Description |
Example |
| configName |
string |
Yes |
The Logtail pipeline configuration name. |
test-config |
Request parameters
|
Parameter |
Type |
Required |
Description |
Example |
| project |
string |
Yes |
The project name. |
test-project |
| body |
object |
No |
The Logtail pipeline configuration content. |
|
| aggregators |
array<object> |
No |
The list of aggregation plug-ins. Important This parameter takes effect only when extended processing plug-ins are used. A maximum of one aggregation plug-in is allowed. |
|
|
object |
No |
The aggregation plug-in. |
||
| configName |
string |
Yes |
The configuration name. Important The value must be the same as the outer configName parameter. |
test-config |
| flushers |
array<object> |
Yes |
The list of output plug-ins. Important Only one Simple Log Service output plug-in is allowed. |
|
|
object |
No |
The output plug-in. |
{ "Type": "flusher_sls", "Logstore": "test" } |
|
| global |
object |
No |
The general configurations. |
|
| inputs |
array<object> |
Yes |
The list of input plug-ins. Important Only one input plug-in can be configured. |
|
|
object |
No |
The input plug-in. Note
For file input plug-in parameters, refer to File plug-in. For other input plug-in parameters, refer to Processing plug-ins. |
{ "Type": "input_file", "FilePaths": ["/var/log/*.log"] } |
|
| logSample |
string |
No |
The log sample. Multiple log entries are supported. |
2022-06-14 11:13:29.796 | DEBUG | __main__: |
| processors |
array<object> |
No |
The list of processing plug-ins. Note
Processing plug-ins are classified into native processing plug-ins and extended processing plug-ins. For more information, refer to Processing plug-ins. Important
Note
|
|
|
object |
No |
The processing plug-in. Note
For more information, refer to Processing plug-ins. |
{ "Type": "processor_parse_json_native", "SourceKey": "content" } |
|
| task |
object |
No |
General configurations
| Parameter | Type | Required | Default value | Example | Description |
| TopicType | string | No | Empty | filepath | The topic type. Valid values:
|
| TopicFormat | string | No. Required when TopicType is set to filepath or custom. | / | /var/log/(.*).log | The topic format. |
| EnableTimestampNanosecond | bool | No | false | false | Specifies whether to enable nanosecond precision for timestamps. |
| PipelineMetaTagKey | object | No | Empty | {"HOST_NAME":"__hostname__"} | Important Supported only in LoongCollector 3.0.10 and later.
|
Input plug-ins.
File input plug-in
| Parameter | Type | Required | Default value | Example | Description |
| Type | string | Yes | / | input_file | The plug-in type. Set the value to input_file. |
| FilePaths | [string] | Yes | / | ["/var/log/*.log"] | The list of log file paths to collect (only one path is supported). The * and ** wildcards are supported in the path. The ** wildcard can appear only once and can be used only before the file name. |
| MaxDirSearchDepth | uint | No | 0 | 0 | The maximum directory depth matched by the ** wildcard in the file path. This parameter takes effect only when the log path contains the ** wildcard. Valid values: 0 to 1000. |
| ExcludeFilePaths | [string] | No | Empty | ["/home/admin/*.log"] | The file path blocklist. Paths must be absolute paths. The * wildcard is supported. |
| ExcludeFiles | [string] | No | Empty | ["app*.log", "password"] | The file name blocklist. The * wildcard is supported. |
| ExcludeDirs | [string] | No | Empty | ["/home/admin/dir1", "/home/admin/dir2*"] | The directory blocklist. Paths must be absolute paths. The * wildcard is supported. |
| FileEncoding | string | No | utf8 | utf8 | The file encoding format. Valid values: utf8 and gbk. |
| TailSizeKB | uint | No | 1024 | 1024 | The size from the end of the file at which collection starts when the configuration takes effect for the first time. If the file size is smaller than this value, collection starts from the beginning. Valid values: 0 to 10485760 KB. |
| Multiline | object | No | Empty | / | The multiline aggregation options. |
| Multiline.Mode | string | No | custom | custom | The multiline aggregation mode. Valid values: custom and JSON. |
| Multiline.StartPattern | string | Required when Multiline.Mode is set to custom | Empty | \d+-\d+-\d+.* | The regular expression for the line start. |
| EnableContainerDiscovery | bool | No | false | true | Specifies whether to enable container discovery. This parameter takes effect only when Logtail runs in DaemonSet mode and the file path to collect is a path inside a container. |
| ContainerFilters | object | No | Empty | / | The container filter options. Multiple options have an AND relationship. This parameter takes effect only when EnableContainerDiscovery is set to true. |
| ContainerFilters.K8sNamespaceRegex | string | No | Empty | default | For containers deployed in a Kubernetes environment, specifies the namespace condition of the pod to which the target container belongs. If this parameter is not specified, all containers are collected. Regular expression matching is supported. |
| ContainerFilters.K8sPodRegex | string | No | Empty | test-pod | For containers deployed in a Kubernetes environment, specifies the pod name condition of the target container. If this parameter is not specified, all containers are collected. Regular expression matching is supported. |
| ContainerFilters.IncludeK8sLabel | map | No | Empty | / | For containers deployed in a Kubernetes environment, specifies the pod label conditions of the target containers. Multiple conditions have an OR relationship. If this parameter is not specified, all containers are collected. Regular expression matching is supported. The key in the map is the pod label name, and the value is the pod label value:
|
| ContainerFilters.ExcludeK8sLabel | map | No | Empty | / | For containers deployed in a Kubernetes environment, specifies the pod label conditions for containers to exclude from collection. Multiple conditions have an OR relationship. If this parameter is not specified, all containers are collected. Regular expression matching is supported. The key in the map is the pod label name, and the value is the pod label value:
|
| ContainerFilters.K8sContainerRegex | string | No | Empty | test-container | For containers deployed in a Kubernetes environment, specifies the container name condition. If this parameter is not specified, all containers are collected. Regular expression matching is supported. |
| ContainerFilters.IncludeEnv | map | No | Empty | / | Specifies the environment variable conditions of the target containers. Multiple conditions have an OR relationship. If this parameter is not specified, all containers are collected. Regular expression matching is supported. The key in the map is the environment variable name, and the value is the environment variable value:
|
| ContainerFilters.ExcludeEnv | map | No | Empty | / | Specifies the environment variable conditions for containers to exclude from collection. Multiple conditions have an OR relationship. If this parameter is not specified, all containers are collected. Regular expression matching is supported. The key in the map is the environment variable name, and the value is the environment variable value:
|
| ContainerFilters.IncludeContainerLabel | map | No | Empty | / | Specifies the container label conditions of the target containers. Multiple conditions have an OR relationship. If this parameter is not specified, the default value is empty, which indicates that all containers are collected. Regular expression matching is supported. The key in the map is the container label name, and the value is the container label value:
|
| ContainerFilters.ExcludeContainerLabel | map | No | Empty | / | Specifies the container label conditions for containers to exclude from collection. Multiple conditions have an OR relationship. If this parameter is not specified, the default value is empty, which indicates that all containers are collected. Regular expression matching is supported. The key in the map is the container label name, and the value is the container label value:
|
| ExternalK8sLabelTag | map | No | Empty | / | For containers deployed in a Kubernetes environment, specifies additional tags related to pod labels to add to logs. The key in the map is the pod label name, and the value is the corresponding tag name. For example, if you add app: k8s\_label\_app to the map and the pod contains the label app=serviceA, this information is added to the log as a tag, that is, the field __tag__:k8s_label_app: serviceA is added. If the pod does not contain the app label, an empty field __tag__:k8s_label_app: is added. |
| ExternalEnvTag | map | No | Empty | / | For containers deployed in a Kubernetes environment, specifies additional tags related to container environment variables to add to logs. The key in the map is the environment variable name, and the value is the corresponding tag name. For example, if you add VERSION: env\_version to the map and the container contains the environment variable VERSION=v1.0.0, this information is added to the log as a tag, that is, the field __tag__:env_version: v1.0.0 is added. If the container does not contain the VERSION environment variable, an empty field __tag__:env_version: is added. |
| CollectingContainersMeta | bool | No | false | true | Specifies whether to enable container metadata preview. |
| AppendingLogPositionMeta | bool | No | false | false | Specifies whether to add metadata about the file to which the log entry belongs, including the __tag__:__inode__ field and the __file_offset__ field. |
| AllowingIncludedByMultiConfigs | bool | No | false | false | Specifies whether to allow the current configuration to collect files that are already matched by other configurations. |
| Tags | object | No | Empty | {"FileInodeTagKey":"__inode__"} | Important Supported only in LoongCollector 3.0.10 and later.
|
| FileOffsetKey | string | No | Empty | __file_offset__ | Important Supported only in LoongCollector 3.0.10 and later. |
Container stdout (legacy)
| Parameter | Type | Required | Default value | Example | Description |
| Type | string | Yes | / | service_docker_stdout | The plug-in type. Set the value to service_docker_stdout. |
| Stdout | Boolean | No | true | true | Specifies whether to collect standard output (stdout). |
| Stderr | Boolean | No | true | true | Specifies whether to collect standard error output (stderr). |
| StartLogMaxOffset | Integer | No | 128×1024 | 131072 | The length of historical data to trace back during the first collection, in bytes. We recommend a value in the range of [131072, 1048576]. |
| IncludeLabel | Map, where LabelKey and LabelValue are of the String type | No | Empty | The container label allowlist, which specifies the containers to collect. The default value is empty, which indicates that stdout of all containers is collected. To configure the container label allowlist, LabelKey is required and LabelValue is optional.
Multiple allowlist entries have an OR relationship. A container is matched if its labels satisfy any allowlist entry. | |
| ExcludeLabel | Map, where LabelKey and LabelValue are of the String type | No | Empty | The container label blocklist, which specifies the containers to exclude from collection. The default value is empty, which indicates that no containers are excluded. To configure the container label blocklist, LabelKey is required and LabelValue is optional.
Multiple blocklist entries have an OR relationship. A container is excluded if its labels satisfy any blocklist entry. | |
| IncludeEnv | Map, where EnvKey and EnvValue are of the String type | No | Empty | The environment variable allowlist, which specifies the containers to collect. The default value is empty, which indicates that stdout of all containers is collected. To configure the environment variable allowlist, EnvKey is required and EnvValue is optional.
| |
| ExcludeEnv | Map, where EnvKey and EnvValue are of the String type | No | Empty | The environment variable blocklist, which specifies the containers to exclude from collection. The default value is empty, which indicates that no containers are excluded. To configure the environment variable blocklist, EnvKey is required and EnvValue is optional.
Multiple blocklist entries have an OR relationship. A container is excluded if its environment variables satisfy any key-value pair. | |
| IncludeK8sLabel | Map, where LabelKey and LabelValue are of the String type | No | Empty | Specifies the containers to collect by using a Kubernetes Label (defined in template.metadata) allowlist. To configure the Kubernetes Label allowlist, LabelKey is required and LabelValue is optional.
Multiple allowlist entries have an OR relationship. A container is matched if its Kubernetes Labels satisfy any allowlist entry. | |
| ExcludeK8sLabel | Map, where LabelKey and LabelValue are of the String type | No | Empty | Specifies the containers to exclude from collection by using a Kubernetes Label (defined in template.metadata) blocklist. To configure the Kubernetes Label blocklist, LabelKey is required and LabelValue is optional.
Multiple blocklist entries have an OR relationship. A container is excluded if its Kubernetes Labels satisfy any blocklist entry. | |
| K8sNamespaceRegex | String | No | Empty | ^(default|nginx)$ | Specifies the containers to collect by namespace name. Regular expression matching is supported. For example, setting this parameter to ^(default|nginx)$ matches all containers in the nginx and default namespaces. |
| K8sPodRegex | String | No | Empty | ^(nginx-log-demo.*)$ | Specifies the containers to collect by pod name. Regular expression matching is supported. For example, setting this parameter to ^(nginx-log-demo.*)$ matches all containers in pods whose names start with nginx-log-demo. |
| K8sContainerRegex | String | No | Empty | ^(container-test)$ | Specifies the containers to collect by container name (the Kubernetes container name is defined in spec.containers). Regular expression matching is supported. For example, setting this parameter to ^(container-test)$ matches all containers named container-test. |
Data processing parameters
| Parameter | Type | Required | Default value | Example | Description |
| BeginLineRegex | String | No | Empty | The regular expression used to match the beginning of a line. If this parameter is empty, single-line mode is used. If the expression matches the beginning of a line, the line is treated as a new log entry. Otherwise, the line is appended to the previous log entry. | |
| BeginLineCheckLength | Integer | No | Empty | The length for matching the beginning of a line. Unit: bytes. Default value: 10 × 1024 bytes. If the regular expression for matching the beginning of a line can be determined within the first N bytes, set this parameter to improve matching efficiency. | |
| BeginLineTimeoutMs | Integer | No | Empty | The timeout period for matching the beginning of a line. Unit: milliseconds. Default value: 3000 milliseconds. If no new log entry appears within 3000 milliseconds, the matching ends and the last log entry is uploaded to Simple Log Service. | |
| MaxLogSize | Integer | No | Empty | The maximum length of a log entry. Default value: 0. Unit: bytes. Default value: 512 × 1024 bytes. If the length of a log entry exceeds this value, the system stops searching for the beginning of the next line and directly uploads the log entry. | |
| ExternalK8sLabelTag | Map, in which LabelKey and LabelValue are of the String type | No | Empty | After you configure Kubernetes Label tags (defined in template.metadata), iLogtail adds Kubernetes Label-related fields to logs. For example, if you set LabelKey to app and LabelValue to | |
| ExternalEnvTag | Map, in which EnvKey and EnvValue are of the String type | No | Empty | After you configure container environment variable tags, iLogtail adds container environment variable-related fields to logs. For example, if you set EnvKey to |
Response elements
|
Element |
Type |
Description |
Example |
None defined.
Examples
Success response
JSON format
{}
Error codes
See Error Codes for a complete list.
Release notes
See Release Notes for a complete list.