Simple Log Service:Collect logs from standard Docker containers

Step 1: Deploy a Logtail container

1. Run the following command to pull the Logtail image. Replace registry.cn-hangzhou.aliyuncs.com based on your business scenario. For information about regions, see Install Logtail on a Linux server. If your server resides in a virtual private cloud (VPC), you must replace registry with registry-vpc.

   docker pull registry.cn-hangzhou.aliyuncs.com/log-service/logtail

2. Run the following command to start a Logtail container:

   docker run --privileged -d \
       -v /:/logtail_host:ro \
       -v /var/run/docker.sock:/var/run/docker.sock \
       --env ALIYUN_LOGTAIL_CONFIG=/etc/ilogtail/conf/${your_region_name}/ilogtail_config.json \
       --env ALIYUN_LOGTAIL_USER_ID=${your_aliyun_user_id} \
       --env ALIYUN_LOGTAIL_USER_DEFINED_ID=${your_machine_group_user_defined_id} \
       registry.cn-hangzhou.aliyuncs.com/log-service/logtail

   Parameter	Description
   --privileged	You must configure the --privileged parameter. Otherwise, the container text file busy error may occur when you delete other containers. For more information, see Bug 1468249, Bug 1441737, and issue 34538. For CentOS 7.4 or later except CentOS 8.0, you can remove the --privileged parameter and run the following command to set the fs.may_detach_mounts parameter to 1: echo fs.may_detach_mounts=1 | sudo tee /etc/sysctl.d/may_detach_mounts.conf sudo sysctl -p /etc/sysctl.d/may_detach_mounts.conf sysctl fs.may_detach_mounts
   ${your_region_name}	The ID of the region where your project resides and the type of the network that your project uses. For more information about regions, see Install Logtail on a Linux server. For more information about network types, see Select a network type. If your project uses the Internet, specify the value in the region-internet format. For example, if your project resides in the China (Hangzhou) region, set the parameter to cn-hangzhou-internet. If your project uses an Alibaba Cloud internal network, specify the value in the region format. For example, if your project resides in the China (Hangzhou) region, set the parameter to cn-hangzhou.
   ${your_aliyun_user_id}	The ID of the Alibaba Cloud account for which Simple Log Service is activated. For more information, see Obtain the ID of the Alibaba Cloud account for which Simple Log Service is activated.
   ${your_machine_group_user_defined_id}	The custom identifier of your machine group. Example: log-docker-demo. Important The custom identifier must be unique in the region where your project resides. When you create a machine group in Step 2, you must enter the custom identifier that you specified in the current step in the Custom Identifier field.

   Important

   You can configure custom settings for the startup parameters of the Logtail container only if the following conditions are met:

   • The following environment variables are configured: ALIYUN_LOGTAIL_USER_DEFINED_ID, ALIYUN_LOGTAIL_USER_ID, and ALIYUN_LOGTAIL_CONFIG.

   • The /var/run directory of the host on which the Logtail container runs is mounted on the /var/run directory of the Logtail container.

   • The root directory of the host is mounted on the /logtail_host directory of the Logtail container.

   • If the The parameter is invalid : uuid=none error is contained in the /usr/local/ilogtail/ilogtail.LOG log file, you must create a file named product_uuid on the host. Then, you must enter a valid UUID in the created file, such as 169E98C9-ABC0-4A92-B1D2-AA6239C0D261, and mount the file on the /sys/class/dmi/id/product_uuid directory of the Logtail container.

Step 2: Create a Logtail configuration

1. Log on to the Simple Log Service console. On the page that appears, click the Quick Data Import card on the right side. In the Import Data dialog box, click the Self-managed Open Source/Commercial Software tab. On this tab, click Docker File - Container or Docker Stdout and Stderr - Container.

   

2. Select the project and Logstore that you want to manage.Then, click Next.

   

3. Create a machine group. If a machine group is available, go to Step 5.

   

4. In the panel that appears after you click Create Machine Group, select Custom Identifier, enter a custom identifier in the Custom Identifier field, and then click OK. The custom identifier must be the same as the custom identifier that you specified in Step 1. Example: log-docker-demo.

   

5. Select the machine group in the Source Server Groups section and move the machine group to the Applied Server Groups section. Then, click Next.

6. In the Logtail Configuration step, configure parameters in the Global Configurations, Input Configurations, and Processor Configurations sections. For more information, see Collect text logs from servers. If you want to parse logs in modes such as regex mode and JSON mode, you can set the Processing Method parameter to Processors and click +Add Processor in the Processor Configurations section. For more information, see Logtail plug-ins overview. After you configure the parameters, click Next.

7. In the Query and Analysis Configurations step, create indexes. For more information, see Create indexes.

Default fields

Docker stdout and stderr

Docker text logs

Related operations

View the status of Logtail

You can run the docker exec ${logtail_container_id} /etc/init.d/ilogtaild status command to view the status of Logtail.

View the version number, IP address, and startup time of Logtail

You can run the docker exec ${logtail_container_id} cat /usr/local/ilogtail/app_info.json command to view the information about Logtail.

View the operational logs of Logtail

[sudo@iZb****exh2Z ilogtail]# docker exec a287de895e40 tail -n 5 /usr/local/ilogtail/ilogtail.LOG
[2018-02-06 08:13:35.721864]    [INFO]    [8]    [build/release64/sls/ilogtail/LogtailPlugin.cpp:104]    logtail plugin Resume:start
[2018-02-06 08:13:35.722135]    [INFO]    [8]    [build/release64/sls/ilogtail/LogtailPlugin.cpp:106]    logtail plugin Resume:success
[2018-02-06 08:13:35.722149]    [INFO]    [8]    [build/release64/sls/ilogtail/EventDispatcher.cpp:369]    start add existed check point events, size:0
[2018-02-06 08:13:35.722155]    [INFO]    [8]    [build/release64/sls/ilogtail/EventDispatcher.cpp:511]    add existed check point events, size:0    cache size:0    event size:0    success count:0
[2018-02-06 08:13:39.725417]    [INFO]    [8]    [build/release64/sls/ilogtail/ConfigManager.cpp:3776]    check container path update flag:0    size:1

start umount useless mount points, /shm$|/merged$|/mqueue$
umount: /logtail_host/var/lib/docker/overlay2/3fd0043af174cb0273c3c7869500fbe2bdb95d13b1e110172ef57fe840c82155/merged: must be superuser to unmount
umount: /logtail_host/var/lib/docker/overlay2/d5b10aa19399992755de1f85d25009528daa749c1bf8c16edff44beab6e69718/merged: must be superuser to unmount
umount: /logtail_host/var/lib/docker/overlay2/5c3125daddacedec29df72ad0c52fac800cd56c6e880dc4e8a640b1e16c22dbe/merged: must be superuser to unmount
......
xargs: umount: exited with status 255; aborting
umount done
start logtail
ilogtail is running
logtail status:
ilogtail is running

Restart Logtail

To restart Logtail, use the following sample code:

[sudo@iZb****xh2Z ilogtail]# docker exec a287de895e40 /etc/init.d/ilogtaild stop
kill process Name: ilogtail pid: 7
kill process Name: ilogtail pid: 8
stop success
[sudo@iZb****xh2Z ilogtail]# docker exec a287de895e40 /etc/init.d/ilogtaild start
ilogtail is running

Troubleshooting

For more information about how to troubleshoot the errors that may occur when you use Logtail to collect logs from Docker containers, see What do I do if errors occur when I collect logs from containers?