How to collect multiple copies of logs from a file - Simple Log Service

By default, a file can match only one Logtail configuration for collection. This topic describes how to collect multiple copies of logs in a file.

Background information

If a file matches multiple Logtail configurations for collection, the usage of CPU, memory, disk I/O, and network I/O resources greatly increases. This affects the performance of other services that are deployed on the same server as Logtail. Therefore, a file can match only one Logtail configuration for collection in default scenarios.

Solutions

Store logs in different logstores

You do not need to create multiple Logtail configurations. Use the data transformation feature of Simple Log Service (SLS) to replicate logs. This prevents the performance of other services on the same server from being affected. For more information, see Replicate data from a logstore.

Create multiple Logtail configurations

Server scenarios

Create a symbolic link for a directory
Create a symbolic link for the directory that contains the log file. Use the original path in one Logtail configuration and the symbolic link path in another. For example, run the following command to create the symbolic link /home/log/nginx/link_log for the /home/log/nginx/log directory. Then read the files in the directory using the symbolic link.
```
ln -s /home/log/nginx/log /home/log/nginx/link_log
```
Add settings in Logtail configurations to forcefully collect logs
When you create a Logtail configuration, enable Allow File to Be Collected for Multiple Times in the Input Configurations section. For more information, see Collect text logs from servers and Manually install Logtail to collect text logs from servers.

Container scenarios

For more information about how to collect container logs, see Collect text logs from an ACK cluster in DaemonSet mode and Collect container logs (standard output and files) from a cluster using the console.

Console
When you create a Logtail configuration, enable Allow File to Be Collected for Multiple Times in the Input Configurations section.
CRD
Create multiple custom resource definitions (CRDs).