Docker events include all interactive events of objects such as containers, images, plug-ins, networks, and volumes. This topic describes how to configure Logtail in the Log Service console to collect Docker events.
Prerequisites
Logtail V0.16.18 or later is installed on a Linux server. For more information, see Install Logtail on a Linux server.Limits
- Logtail that runs on containers or hosts must be authorized to access the
/var/run/docker.sock
file.For information about how to use Logtail to collect Kubernetes logs, see Collect Kubernetes logs. For information about how to collect standard container logs, see Collect logs from standard Docker containers.
- When Logtail is restarted or stopped, container events are not collected.
Scenarios
- Monitor the start and stop events of all containers, and trigger alerts when core containers stop running.
- Collect all container events for auditing, security analysis, and troubleshooting.
- Monitor all image pulling events, and trigger an alert if an image is pulled from an invalid path.
Procedure
- Log on to the Log Service console.
- In the Import Data section, select Custom Data Plug-in.
- Select the project and Logstore. Then, click Next.
- In the Create Machine Group step, create a machine group.
- If a machine group is available, click Use Existing Machine Groups.
- If no machine groups are available, perform the following steps to create a machine group. In this example, an Elastic Compute Service (ECS) instance is used.
- On the ECS Instances tab, select Manually Select Instances. Then, select the ECS instance that you want to use and click Create.
For more information, see Install Logtail on ECS instances.
Important If you want to collect logs from an ECS instance that belongs to a different Alibaba Cloud account, a server in an on-premises data center, or a server of a third-party cloud service provider, you must manually install Logtail. For more information, see Install Logtail on a Linux server. After you manually install Logtail, you must configure a user identifier for the server. For more information, see Configure a user identifier. - After Logtail is installed, click Complete Installation.
- In the Create Machine Group step, configure the Name parameter and click Next.
Log Service allows you to create IP address-based machine groups and custom identifier-based machine groups. For more information, see Create an IP address-based machine group and Create a custom identifier-based machine group.
- On the ECS Instances tab, select Manually Select Instances. Then, select the ECS instance that you want to use and click Create.
- Select the new machine group from Source Server Groups and move the machine group to Applied Server Groups. Then, click Next. Important If you apply a machine group immediately after you create the machine group, the heartbeat status of the machine group may be FAIL. This issue occurs because the machine group is not connected to Log Service. To resolve this issue, you can click Automatic Retry. If the issue persists, see What do I do if no heartbeat connections are detected on Logtail?
- In the Specify Data Source step, configure the Config Name and Plug-in Config parameters. Then, click Next.
- inputs specifies the collection configurations of your data source. This parameter is required. Important You can specify only one type of data source in the inputs parameter.
- processors specifies the processing configurations that are used to parse data. You can extract fields, extract log time, desensitize data, and filter logs. This parameter is optional. You can specify one or more processing methods. For more information, see Overview.
{ "inputs": [ { "detail": {}, "type": "service_docker_event" } ] }
Parameter Type Required Description type string Yes The type of the data source. Set the value to service_docker_event. EventQueueSize int No The maximum number of events in the event queue. Default value: 10. - inputs specifies the collection configurations of your data source. This parameter is required.
- Preview data, configure indexes, and then click Next. By default, full-text indexing is enabled for Log Service. You can also configure field indexes based on collected logs in manual mode or automatic mode. To configure field indexes in automatic mode, click Automatic Index Generation. This way, Log Service automatically creates field indexes. For more information, see Create indexes.Important If you want to query and analyze logs, you must enable full-text indexing or field indexing. If you enable both full-text indexing and field indexing, the system uses only field indexes.
- Click Log Query. You are redirected to the query and analysis page of your Logstore. You must wait approximately 1 minute for the indexes to take effect. Then, you can view the collected logs on the Raw Logs tab. For more information, see Query and analyze logs.
Troubleshooting
If no data is displayed on the preview page or query page after logs are collected by using Logtail, you can troubleshoot the errors based on the instructions that are provided in What do I do if errors occur when I use Logtail to collect logs?
Sample logs
This section provides sample Docker events.
- Example 1: image pulling event
__source__: 10.10.10.10 __tag__:__hostname__: logtail-ds-77brr __topic__: _action_: pull _id_: registry.cn-hangzhou.aliyuncs.com/ringtail/eventer:v1.6.1.3 _time_nano_: 1547910184047414271 _type_: image name: registry.cn-hangzhou.aliyuncs.com/ringtail/eventer
- Example 2: container destruction event in Kubernetes
__source__: 10.10.10.10 __tag__:__hostname__: logtail-ds-xnvz2 __topic__: _action_: destroy _id_: af61340b0ac19e6f5f32be672d81a33fc4d3d247bf7dbd4d3b2c030b8bec4a03 _time_nano_: 1547968139380572119 _type_: container annotation.kubernetes.io/config.seen: 2019-01-20T15:03:03.114145184+08:00 annotation.kubernetes.io/config.source: api annotation.scheduler.alpha.kubernetes.io/critical-pod: controller-revision-hash: 2630731929 image: registry-vpc.cn-hangzhou.aliyuncs.com/acs/pause-amd64:3.0 io.kubernetes.container.name: POD io.kubernetes.docker.type: podsandbox io.kubernetes.pod.name: logtail-ds-44jbg io.kubernetes.pod.namespace: kube-system io.kubernetes.pod.uid: 6ddcf598-1c81-11e9-9ddf-00163e0c7cbe k8s-app: logtail-ds kubernetes.io/cluster-service: true name: k8s_POD_logtail-ds-44jbg_kube-system_6ddcf598-1c81-11e9-9ddf-00163e0c7cbe_0 pod-template-generation: 9 version: v1.0
The following table describes the fields in a Docker event. For more information, see docker events.
Field | Description |
---|---|
_type_ | The type of the resource. Example: container or image. |
_action_ | The type of the action. Example: destroy or status. |
_id_ | The unique ID of the event. |
_time_nano_ | The timestamp of the event. |