Use the console to collect Kubernetes text logs in DaemonSet mode

Features

Limits

• Stop policy: When a container is stopped and Logtail detects the die event on the container, Logtail stops collecting logs from the container. In this case, if a collection delay occurs, some logs that are generated before the stop action may be lost.
• Docker storage driver: Only overlay and overlay2 are supported. For other storage drivers, you must mount the log directory on the local host.
• Symbolic link: Logtail cannot access the symbolic link of a container. Set the collection directory to an actual path.

Configure log collection

1. Log on to the Log Service console.
2. In the Import Data section, click Kubernetes - Object.
3. Select the created project and Logstore and click Next.
4. In the Create Machine Group step, create a machine group as prompted, and then click Complete Installation.

   If a machine group is available, click Use Existing Machine Groups.

5. Select and move the destination machine group from Source Server Groups to Applied Server Groups, and then click Next.
   Notice If you want to apply a machine group immediately after it is created, the heartbeat status of the machine group may be FAIL. This issue occurs because the machine group has not been connected to Log Service. In this case, you can click Automatic Retry. If the issue persists, see What can I do if the Logtail client has no heartbeat?
6. In the Logtail Config step, specify the data source, and then click Next. The following table describes the parameters.

   Parameter	Required	Description
   Docker File	Yes	Checks whether the file that is collected from the specified data source is a Docker file.
   Label Whitelist	No	If you specify a label whitelist, the LabelKey parameter must be specified. If the value of the LabelValue parameter is not empty, logs are collected from the containers whose label key-value pairs match the specified key-value pairs. If the value of the LabelValue parameter is empty, logs are collected from the containers whose label keys match the specified keys. Note Key-value pairs are associated by the OR operator. If a label key-value pair of a container matches one of the specified key-value pairs, logs of the container are collected. By default, the value of the LabelValue parameter is a string. Logs are collected from the container whose name matches the value of the LabelValue parameter. If you use a regular expression to specify the value of the LabelValue parameter, logs are collected from the containers whose names match the regular expression. For example, if you specify the parameter value that starts with a caret (^) and ends with a dollar sign ($) such as ^(kube-system|istio-system)$, logs are collected from the container named kube-system and the container named istio-system. Do not specify duplicate values for the LabelKey parameter. Otherwise, the existing value of the LabelValue parameter is overwritten.
   Label Blacklist	No	If you specify a label blacklist, the LabelKey parameter must be specified. If the value of the LabelValue parameter is not empty, logs are not collected from the containers whose label key-value pairs match the specified key-value pairs. If the value of the LabelValue parameter is empty, logs are not collected from the containers whose label keys match the specified keys. Note Key-value pairs are associated by the OR operator. If a label key-value pair of a container matches one of the specified key-value pairs, logs of the container are collected. By default, the value of the LabelValue parameter is a string. Logs are collected from the container whose name matches the value of the LabelValue parameter. If you use a regular expression to specify the value of the LabelValue parameter, logs are collected from the containers whose names match the regular expression. For example, if you specify the parameter value that starts with a caret (^) and ends with a dollar sign ($) such as ^(kube-system|istio-system)$, logs are collected from the container named kube-system and the container named istio-system. Do not specify duplicate values for the LabelKey parameter. Otherwise, the existing value of the LabelValue parameter is overwritten.
   Environment Variable Whitelist	No	If you specify an environment variable whitelist, the EnvKey parameter must be specified. If the value of the EnvValue parameter is not empty, logs are collected from the containers whose environment variable key-value pairs match the specified key-value pairs. If the value of the EnvValue parameter is empty, logs are collected from the containers whose environment variable keys match the specified keys. Note Key-value pairs are associated by the OR operator. If an environment variable key-value pair of a container matches one of the specified key-value pairs, logs of the container are collected. By default, the value of the EnvValue parameter is a string. Logs are collected from the container whose name matches the value of the EnvValue parameter. If you use a regular expression to specify the value of the EnvValue parameter, logs are collected from the containers whose names match the regular expression. For example, if you specify the value of the EnvValue parameter that starts with a caret (^) and ends with a dollar sign ($) such as ^(kube-system|istio-system)$, logs are collected from the container named kube-system and the container named istio-system.
   Environment Variable Blacklist	No	If you specify an environment variable blacklist, the EnvKey parameter must be specified. If the value of the EnvValue parameter is not empty, logs are not collected from the containers whose environment variable key-value pairs match the specified key-value pairs. If the value of the EnvValue parameter is empty, logs are not collected from the containers whose environment variable keys match the specified keys. Note Key-value pairs are associated by the OR operator. If an environment variable key-value pair of a container matches one of the specified key-value pairs, logs of the container are collected. By default, the value of the EnvValue parameter is a string. Logs are collected from the container whose name matches the value of the EnvValue parameter. If you use a regular expression to specify the value of the EnvValue parameter, logs are collected from the containers whose names match the regular expression. For example, if you specify the value of the EnvValue parameter that starts with a caret (^) and ends with a dollar sign ($) such as ^(kube-system|istio-system)$, logs are collected from the container named kube-system and the container named istio-system.
   Other parameters	N/A	For more information about other parameters, see Overview. Note By default, the root directory of the host is mounted on the /logtail_host directory of the Logtail container. When you configure the directory for log collection, you must add the container directory as the prefix of the log path. For example, to collect data from the /home/logs/app_log/ directory of the host, you must set the log path to /logtail_host/home/logs/app_log/.

   Note

   • A namespace and a container name in a Kubernetes cluster can be mapped to Docker labels. The value of the LabelKey parameter for a namespace is io.kubernetes.pod.namespace. The value of the LabelKey parameter for a container name is io.kubernetes.container.name. For example, the namespace of the pod that you have created is backend-prod and the container name is worker-server. In this case, you can set the key-value pair of a whitelist label to io.kubernetes.pod.namespace : backend-prod or io.kubernetes.container.name : worker-server. Then, you can collect logs from only the worker-server container.
   • In a Kubernetes cluster, we recommend that you specify only the io.kubernetes.pod.namespace and io.kubernetes.container.name labels. You can also specify the Environment Variable Whitelist or Environment Variable Blacklist parameter based on your business requirements.
7. In the Configure Query and Analysis step, specify the indexes, and then click Next.
   Indexes are created by default. You can modify the indexes as needed.

Configuration examples

• Configure environment variables

  Collect the logs of the containers that meet the following conditions: The environment variables include NGINX_PORT_80_TCP_PORT=80 and exclude POD_NAMESPACE=kube-system, the log file path is /var/log/nginx/access.log, and logs are parsed in simple mode.

  
  The following figure shows the configuration of a data source. For more information about log collection modes, see Collect logs in full regex mode, Collect logs in NGINX mode, and Collect logs in delimiter mode.
• Label configuration
  Collect the logs of the containers that meet the following conditions: The container labels include io.kubernetes.container.name=nginx, the log file path is /var/log/nginx/access.log, and logs are parsed in simple mode.
  The following figure shows the configuration of a data source. For more information about log collection modes, see Collect logs in full regex mode, Collect logs in NGINX mode, and Collect logs in delimiter mode.

Default fields