Generate Logtail CRD and API scripts - Simple Log Service

The Logtail configuration generator automatically builds AliyunPipelineConfig CRD definitions and API parameter scripts for Logtail collection tasks, eliminating manual configuration.

Procedure

Log on to the Simple Log Service console.
In the Projects section, click the one you want.
In the left-side navigation pane, click > CRD Generator.
On the CRD Generator page, select a collection scenario and configure Logtail in the left panel. The Preview Configurations panel on the right then generates a YAML script for the AliyunPipelineConfig CRD and a JSON object for API calls.
- Collection scenario: The supported scenarios include Text Logs for hosts, Kubernetes - File, and stdout and stderr - New/Old Version.
- Logtail configuration: Click Import Other Configuration to import an existing configuration from the Import Logtail Configuration page, or create a new one.
- Preview Configurations:
  - YAML: Generates a ready-to-use script for creating a Logtail configuration with an AliyunPipelineConfig CRD.
  - JSON: Generates a JSON object for the CreateLogtailPipelineConfig API operation. The key-value pairs map to the parameters and values of the API call.
The left panel includes Global Configurations (Project Name, Logstore Name, Configuration Name, log topic type, and advanced parameters) and Input Configurations. For text logs, select a Logtail deployment mode (DaemonSet or sidecar) and set the file path.
To use Logtail SPL to parse logs in a new configuration, first add a Log Sample. When you Preview the processing results, verify that the correct Project Name and Logstore Name are set.

On the Processing Configurations page, set Processing Mode to SPL, enter a parsing statement such as * | parse-json content in the SPL Statement editor, and click Preview.
Copy the generated YAML or JSON script and create a Logtail configuration for your scenario.

Scenarios

Text logs

CRD-based configuration is not supported for host text logs. The following example uses the generator to create a Logtail configuration through an API call.

Generate API parameters: In the left panel of the Logtail configuration generator page, complete the Logtail configuration or click Import Other Configuration.

Select Text Logs as the collection type. In the left panel, configure Global Configurations (Project Name, Logstore Name, Configuration Name), Input Configurations (type: text logs, File Path: /data/wwwlogs/**/test.log, Max Directory Watch Depth: 0), and Processing Configurations (Plugin Combination mode with the native RegEx Parser plugin). The JSON tab in Preview Configurations displays the complete pipeline, including inputs (input_file), processors (processor_parse_regex_native), and flushers (flusher_sls).
Call an API operation: The generated JSON script corresponds to the parameters for the CreateLogtailPipelineConfig operation. Try the API in the Debugging console.

Kubernetes - file

On the CRD Generator page, click Kubernetes - File.
In the left panel, click Import Other Configuration to use an existing configuration, or create a new one following these documents:
- To collect text logs from an ACK (Container Service for Kubernetes) cluster where Logtail is deployed as a DaemonSet, see Create a Logtail configuration in the Simple Log Service console.
- To collect text logs from a self-managed Kubernetes cluster where Logtail is deployed as a DaemonSet, see Collect container logs from a cluster by using the console (stdout and stderr/files).
Create a Logtail configuration.
AliyunPipelineConfig CRD

Use the YAML script from Preview Configurations to create a Logtail configuration with an AliyunPipelineConfig CRD:
- To collect text logs from an ACK cluster where Logtail is deployed as a DaemonSet, see [Recommended] Use an AliyunPipelineConfig CRD.
- To collect text logs from a self-managed Kubernetes cluster where Logtail is deployed as a DaemonSet, see Collect container logs from a cluster by using the console (stdout and stderr/files).
API

The generated JSON script corresponds to the parameters for the CreateLogtailPipelineConfig operation. Try the API in the Debugging console.

Stdout and stderr - new/old version

On the CRD Generator page, click stdout and stderr - New Version or stdout and stderr - Old Version.
In the left panel, click Import Other Configuration to use an existing configuration, or create a new one following these documents:
- To collect stdout and stderr (old version) from an ACK cluster where Logtail is deployed as a DaemonSet, see Create a Logtail configuration in the Simple Log Service console.
- To collect stdout and stderr (old version) from a self-managed Kubernetes cluster where Logtail is deployed as a DaemonSet, see Create a Logtail configuration in the Simple Log Service console.
Create a Logtail configuration.
AliyunPipelineConfig CRD

Use the YAML script from Preview Configurations to create a Logtail configuration with an AliyunPipelineConfig CRD:
- To collect stdout and stderr (old version) from an ACK cluster where Logtail is deployed as a DaemonSet, see [Recommended] Use an AliyunPipelineConfig CRD.
- To collect stdout and stderr (old version) from a self-managed Kubernetes cluster where Logtail is deployed as a DaemonSet, see
  
  [Recommended] Use an AliyunPipelineConfig CRD.
API

The generated JSON script corresponds to the parameters for the CreateLogtailPipelineConfig operation. Try the API in the Debugging console.