如何采集脚本执行日志 - Simple Log Service - Alibaba Cloud Documentation Center

Create a Logtail collection configuration in the Simple Log Service (SLS) console to collect script execution logs.

Prerequisites

Logtail 1.7.1 or later is installed on your Linux server. For more information, see Install Logtail on a Linux server.

Functionality

Logtail executes a configured script as a specified user to collect the standard output of the script as logs.

Supported script types: bash, shell, python2, and python3.
Specifying a user to run the script lets you limit the resources the script can access.
If a script times out, the system automatically stops it. Logtail does not collect logs from the timed-out execution.

Procedure

Log on to the Simple Log Service console.
In the Import Data section, select Custom Data Plug-in.
Select the destination Project and Logstore, and then click Next.
Create a machine group.
- If you have an existing machine group, click Use Existing Machine Groups.
- If you do not have a machine group, follow these steps. This example uses an ECS instance.
  1. On the ECS Instance tab, select the target ECS instance and click Create.
    
    For more information, see Install Logtail on ECS instances.
    
    Important
    You must manually install Logtail if your server is an ECS instance from a different Alibaba Cloud account, a server from another cloud provider, or a self-managed on-premises server. For more information, see Install Logtail on a Linux server or Install Logtail on a Windows server. After you manually install Logtail, you must configure a user identifier on the server. For more information, see Configure a user identifier.
  2. After the installation is complete, click Complete Installation.
  3. On the Create Machine Group page, enter a Name and click Next.
    
    SLS supports IP address-based machine groups and custom identifier-based machine groups. For more information about the parameters, see Create an IP address-based machine group and Create a custom identifier-based machine group.
Select the target machine group, move it from Source Machine Group to Applied Machine Group, and then click Next.

Important
If you apply a machine group immediately after you create it, the heartbeat status may be FAIL because the connection is not yet established. You can click Automatic Retry. If the issue persists, see Troubleshoot Logtail machine group issues in a host environment.

On the Configure Data Source tab, specify Configuration Name and Plug-in Configuration, and then click Next.

inputs: Required. The data source configuration.

Important
Only one type of data source can be configured in a single inputs.
processors: Optional. The data processing configuration.

If inputs alone does not meet your log parsing requirements, add a processors section in the Plug-in Configuration section to use Logtail plug-ins for data processing. For example, you can extract fields, parse log time, mask data, or filter logs. For more information, see Use native and extension processing plug-ins.

{
    "inputs":[
        {
            "type":"input_command",
            "detail":{
                "ScriptType":"python2",
                "User":"test",
                "ScriptContent":"print(\"test input_command 0\")\nprint(\"test input_command 1\")\nprint(\"test input_command 2\")\nprint(\"test input_command 3\")\nprint(\"test input_command 4\")\nprint(\"test input_command 5\")\nprint(\"test input_command 6\")\nprint(\"test input_command 7\")\nprint(\"test input_command 8\")\nprint(\"test input_command 9\")\n",
                "ContentEncoding":"PlainText",
                "CmdPath":"/usr/bin/python",
                "TimeoutMilliSeconds":1000,
                "IgnoreError":false,
                "Environments":[
                    "DEBUG=true"
                ],
                "IntervalMs":5000
            }
        }
    ]
}

Parameter	Type	Required	Description
type	String	Yes	The data source type. Set to `input_command`.
ScriptType	String	Yes	The script type. Valid values: `bash`, `shell`, `python2`, and `python3`.
User	String	Yes	The user that runs the script. Only non-root users are supported. Note Ensure that the specified user exists on the machine. Follow the principle of least privilege by granting the user read, write, and execute (rwx) permissions only on necessary directories and files.
ScriptContent	String	Yes	The content of the script. The content can be plain text or Base64-encoded. The maximum size is 512 KB.
ContentEncoding	String	No	The encoding format of the script content. Valid values: `PlainText` (Default): The content is plain text and is not encoded. `Base64`: The content is Base64-encoded.
LineSplitSep	String	No	The separator for splitting script output into multiple log entries. If left empty, the entire output is treated as a single log entry.
CmdPath	String	No	The path to the script interpreter. If left empty, a default path is used. `bash`: /usr/bin/bash `shell`: /usr/bin/sh `python2`: /usr/bin/python2 `python3`: /usr/bin/python3
TimeoutMilliSeconds	Integer	No	The script execution timeout in milliseconds. Default: `3000`.
IgnoreError	Boolean	No	Whether to suppress plug-in error logging. Default: `false` (errors are logged).
Environments	[]String	No	Custom environment variables passed to the script, in addition to the operating system defaults.
IntervalMs	Integer	Yes	The interval for executing the script and collecting its output, in milliseconds. Default: `5000`.

Preview the data, configure indexes, and then click Next.

By default, Simple Log Service enables full-text indexing. You can also create field indexes based on the collected logs, or click Automatic Index Generation to create field indexes automatically. For more information, see Create indexes.

Important
If you want to query and analyze logs, you must enable full-text indexing or field indexes. If both are enabled, field indexes take precedence.
Click Query Log to go to the query and analysis page for the Logstore.

Wait about 1 minute for the indexes to take effect. You can then view the collected logs on the Raw Log tab. For more information, see Get started with query and analysis.

Example

Use a python2 script to get the system time.

Logtail collection configuration

{
    "inputs":[
        {
            "type":"input_command",
            "detail":{
                "ScriptType":"python2",
                "User":"test",
                "ScriptContent":"import datetime\nprint(datetime.datetime.now())\n",
                "ContentEncoding":"PlainText",
                "CmdPath":"/usr/bin/python",
                "TimeoutMilliSeconds":1000,
                "IgnoreError":false,
                "Environments":[
                    "DEBUG=true"
                ],
                "IntervalMs":5000
            }
        }
    ]
}

Collected log

content:"2023-08-07 02:17:52.123456"
script_md5:"a9564ebc3289b7a14551baf8ad5ec60a"
__pack_meta__:"1|MTY4OTI1MzExNTU2NTE2MzQ1NA==|2|1"
__topic__:""
__source__:"172.17.0.2"
__tag__:__pack_id__:"D7B7F988051A3019-0"
__tag__:__hostname__:"bc6c7472f257"
__tag__:__client_ip__:"8.218.101.91"
__tag__:__receive_time__:"1691374673"
__time__:"1691374672"

Field descriptions

Field	Description
content	The log data from the script's standard output.
script_md5	The MD5 hash of the script content.

Troubleshooting

If no data appears on the preview or query page after you configure Logtail to collect logs, see Troubleshoot Logtail collection failures.