This topic describes how to create a Logtail configuration in the Simple Log Service console to collect script execution logs.
Prerequisites
Logtail V1.7.1 or later is installed on your Linux server. For more information, see Install Logtail on a Linux server.
Description
You can specify a user and a script in a Logtail configuration. Logtail executes the script as the user to obtain script execution logs.
You can specify one of the following types of scripts: bash, shell, python2, and python3.
You can specify a user to execute the script. This way, you can achieve user-based access control on resources that can be accessed by the script.
If the script execution times out, Logtail automatically stops executing the script. Logtail does not collect the script execution logs.
Procedure
Log on to the Simple Log Service console.
In the Import Data section, select Custom Data Plug-in.
Select a project and a Logstore. Then, click Next.
Create a machine group.
If a machine group is available, click Use Existing Machine Groups.
If no machine groups are available, perform the following steps to create a machine group. In this example, an Elastic Compute Service (ECS) instance is used.
On the ECS Instances tab, select Manually Select Instances. Then, select the ECS instance that you want to use and click Create.
For more information, see Install Logtail on ECS instances.
ImportantIf your server is an ECS instance that belongs to a different Alibaba Cloud account from Simple Log Service, a server in a data center, or a server from a third-party cloud service provider, you must manually install Logtail on the server. For more information, see Install Logtail on a Linux server or Install Logtail on a Windows server. After you manually install Logtail, you must configure a user identifier for the server. For more information, see Configure a user identifier.
After Logtail is installed, click Complete Installation.
In the Create Machine Group step, configure the Name parameter and click Next.
Simple Log Service allows you to create IP address-based machine groups and custom identifier-based machine groups. For more information, see Create an IP address-based machine group and Create a custom identifier-based machine group.
Select the machine group from Source Server Groups and move the machine group to Applied Server Groups. Then, click Next.
ImportantIf you apply a machine group immediately after you create the machine group, the heartbeat status of the machine group may be FAIL. This issue occurs because the machine group is not connected to Simple Log Service. To resolve this issue, you can click Automatic Retry. If the issue persists, troubleshoot the issue by following the instructions provided in How do I troubleshoot an error that is related to a Logtail machine group in a host environment?
In the Specify Data Source step, configure the Config Name and Plug-in Config parameters. Then, click Next.
inputs
is required and is used to configure the data collection settings for the Logtail configuration. You must configure inputs based on your data source.ImportantYou can configure only one type of data source in the
inputs
field.processors
is optional and is used to configure the data processing settings for the Logtail configuration. You can specify one or more processing methods.If your logs cannot be parsed based only on the setting of
inputs
, you can configure processors in the Plug-in Config field to add plug-ins for data processing. For example, you can extract fields, extract log time, mask data, and filter logs. For more information, see Overview.
{ "inputs":[ { "type":"input_command", "detail":{ "ScriptType":"python2", "User":"test", "ScriptContent":"print(\"test input_command 0\")\nprint(\"test input_command 1\")\nprint(\"test input_command 2\")\nprint(\"test input_command 3\")\nprint(\"test input_command 4\")\nprint(\"test input_command 5\")\nprint(\"test input_command 6\")\nprint(\"test input_command 7\")\nprint(\"test input_command 8\")\nprint(\"test input_command 9\")\n", "ContentEncoding":"PlainText", "CmdPath":"/usr/bin/python", "TimeoutMilliSeconds":1000, "IgnoreError":false, "Environments":[ "DEBUG=true" ], "IntervalMs":5000 } } ] }
Parameter
Type
Required
Description
type
string
Yes
The type of the data source. Set the value to
input_command
.ScriptType
string
Yes
The type of the script. Valid values: bash, shell, python2, and python3.
User
string
Yes
The name of the user that is used to execute the script. Only non-root users are supported.
NoteMake sure that the specified user name exists on the server.
We recommend that you grant the user only the rwx permissions on the required directories or files.
ScriptContent
string
Yes
The content of the script. The content can be Base64-encoded or plaintext. The content must be within 524,288 bytes in length.
ContentEncoding
string
No
The text format of the script content. This parameter is optional. Valid values:
PlainText (default)
Base64
LineSplitSep
string
No
The delimiter that is used to separate the output of the script. If you leave this parameter empty, the output of the script is returned as a single data record.
CmdPath
string
No
The path to the script. If you leave this parameter empty, the default path is used. Default paths:
bash: /usr/bin/bash
shell: /usr/bin/sh
python2: /usr/bin/python2
python3: /usr/bin/python3
TimeoutMilliSeconds
string
No
The timeout period for script execution. Default value: 3000. Unit: milliseconds.
IgnoreError
Bool
No
Specifies whether to ignore error logs that are generated when Logtail fails to execute the script. Default value: false. The value false indicates that error logs are not ignored.
Environments
[]string
No
The environment variable. By default, the value of os.Environ() is used. If you configure this parameter, the value of this parameter is used together with the value of os.Environ().
IntervalMs
int
Yes
The intervals at which logs are collected or the script is executed. Default value: 5000. Unit: milliseconds.
Preview data, configure indexes, and then click Next.
By default, full-text indexing is enabled in Simple Log Service. You can configure field indexes based on collected logs in manual or automatic mode. To configure field indexes in automatic mode, click Automatic Index Generation. Simple Log Service automatically creates field indexes. For more information, see Create indexes.
ImportantIf you want to query and analyze logs, you must enable full-text indexing or field indexing. If you enable both full-text indexing and field indexing, the system uses only field indexes.
Click Log Query. Then, you are redirected to the query and analysis page of your Logstore.
You must wait approximately 1 minute for the indexes to take effect. Then, you can view the collected logs on the Raw Logs tab. For more information, see Query and analyze logs.
Example
Obtain the system time by using a python2 script.
Logtail configuration
{ "inputs":[ { "type":"input_command", "detail":{ "ScriptType":"python2", "User":"test", "ScriptContent":"print(\"test input_command 0\")\nprint(\"test input_command 1\")\nprint(\"test input_command 2\")\nprint(\"test input_command 3\")\nprint(\"test input_command 4\")\nprint(\"test input_command 5\")\nprint(\"test input_command 6\")\nprint(\"test input_command 7\")\nprint(\"test input_command 8\")\nprint(\"test input_command 9\")\n", "ContentEncoding":"PlainText", "CmdPath":"/usr/bin/python", "TimeoutMilliSeconds":1000, "IgnoreError":false, "Environments":[ "DEBUG=true" ], "IntervalMs":5000 } } ] }
Collected log
content:"Mon Aug 7 02:17:52 UTC 2023" script_md5:"a9564ebc3289b7a14551baf8ad5ec60a" __pack_meta__:"1|MTY4OTI1MzExNTU2NTE2MzQ1NA==|2|1" __topic__:"" __source__:"172.17.0.2" __tag__:__pack_id__:"D7B7F988051A3019-0" __tag__:__hostname__:"bc6c7472f257" __tag__:__client_ip__:"8.218.101.91" __tag__:__receive_time__:"1691374673" __time__:"1691374672"
Field description
Field
Description
content
The script execution log that is collected.
script_md5
The MD5 hash value of the script content.
Troubleshooting
If no data is displayed on the preview page or query page after logs are collected by using Logtail, you can troubleshoot the errors based on the instructions that are provided in What do I do if errors occur when I use Logtail to collect logs?