By using Logtail, you can configure TCP ports locally to receive syslog data forwarded by syslog agents by means of the TCP protocol. Logtail parses the received data and forwards it to LogHub.
Step 1. Create a Logtail syslog configuration in the Log Service console
- Log on to the Log Service console, and click the target project to enter the Logstore List.
- On the Logstore List page, click the Data Import Wizard icon at the right of the Logstore.
- Select the data source type.
Select syslog in Other Sources and click Next.
- Specify the Configuration Name.
The configuration name can be 3–63 characters long, contain lowercase letters, numbers, hyphens (-), and underscores (_), and must begin and end with a lowercase letter or number.
Note The configuration name cannot be modified after the configuration is created.
- Specify Tag Settings.
For more information, see ~~43759~~.
- Set Advanced Options as needed.
Select whether to enable Local Cache. When the Log Service is unavailable, logs can be cached in the local directory of the machine, and continue to be sent to Log Service after the service recovery. By default, this function is enabled, and at most 1 GB logs can be cached.
- Select the machine group and click
Apply to Machine Group to apply the configuration to the selected machine group.
If you have not created a machine group, you must create one first. For how to create a machine group, see ~~28966~~.
Step 2. Configure Logtail to bring the protocol into effect
ilogtail_config.json file in the Logtail installation directory on the machine. Generally, it is in the
/usr/local/ilogtail/ directory. Modify the syslog configuration as needed.
- Confirm that the syslog function is enabled.
true indicates the function is enabled. false indicates the function is disabled.
"streamlog_open" : true
- Specify the size of the memory pool that syslog uses to receive logs Logtail requests a specified size of memory at one time when started. Configure the size (in MB) according to the memory size of your machine and your actual requirements.
"streamlog_pool_size_in_mb" : 50
- Specify the buffer size (in bytes). You must specify the size of the buffer that Logtail uses when calling the socket io rcv interface.
"streamlog_rcv_size_each_call" : 1024
- Specify the syslog log format.
- Specify the TCP binding address and port. You must specify the TCP binding address and port that Logtail uses to receive syslog data. By default, the binding address is 0.0.0.0 and the binding port is 11111.
"streamlog_tcp_addr" : "0.0.0.0", "streamlog_tcp_port" : 11111
- After the configuration is complete, restart Logtail. To restart Logtail, run the following commands to stop the Logtail client and then start it again.
sudo /etc/init.d/ilogtaild stop sudo /etc/init.d/ilogtaild start
Step 3. Install rsyslog and modify its configurations
Skip this step if you have installed rsyslog on the machine.
- Install rsyslog.
For the installation method, see:
- Modify configurations.
/etc/rsyslog.conf, modify the configurations as needed, for example:
$WorkDirectory /var/spool/rsyslog # where to place spool files $ActionQueueFileName fwdRule1 # unique name prefix for spool files $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) $ActionQueueSaveOnShutdown on # save messages to disk on shutdown $ActionQueueType LinkedList # run asynchronously $ActionResumeRetryCount -1 # infinite retries if host is down # Defines the fields of log data $template ALI_LOG_FMT,"0.1 sys_tag %timegenerated:::date-unixtimestamp% %fromhost-ip% %hostname% %pri-text% %protocol-version% %app-name% %procid% %msgid% %msg:::drop-last-lf%\n" *. * @@10.101.166.173:11111;ALI_LOG_FMT
Note In the template
ALI_LOG_FMT, the value of the second field is
sys_tag. This value must be the same as the one entered in step 1. This configuration indicates that all the ( \*.\* ) syslog data received
\*.\*by this machine is formatted according to the
ALI_LOG_FMTtemplate, and forwarded to 10.101.166.173:11111 by using the TCP protocol. The machine 10.101.166.173 must be in the machine group selected in step 1 and configured according to step 2.
- Start rsyslog.
sudo /etc/init.d/rsyslog restart
Before starting rsyslog, check whether another syslog agent is installed on the machine, such as syslogd, sysklogd, or syslog-ng. If yes, stop that syslog agent.
After completing the preceding three steps, you can collect syslog data on the machine to Log Service.
For more information about syslog collection, and how to format syslog data, see ~~43759~~.