Collect systemd journal logs of the node - Container Service for Kubernetes

In Linux, systemd is the initialization system and service manager that is responsible for managing all services after boot. A critical part of systemd is Journal, which collects and stores system logs. In Kubernetes environments, querying and analyzing systemd journal logs is essential to obtain information on node stability, such as logs from the kubelet and the operating system. Container Service for Kubernetes (ACK) clusters integrate with Simple Log Service. You can install the Logtail component to collect logs by using DaemonSets.

Billing

To use this feature, in addition to fees related to ACK clusters, charges related to Simple Log Service also incur. Simple Log Service operates on a pay-as-you-go basis. For more information, see Simple Log Service billing overview.

Prerequisites

The logtail-ds component version 1.0.0 or later is installed in the ACK cluster. For installation instructions, see Step 1: Install Logtail.
A kubectl client is connected to the ACK cluster. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.

Step 1: Use AliyunLogConfig to configure a collection method

AliyunLogConfig is a specialized custom resource definition (CRD) for Simple Log Service. By using the YAML file for AliyunLogConfig, you can create a corresponding custom resource (CR) that manages the log collection configuration of logtail-ds.

The following sample code uses a Logtail configuration named systemd-journal-log-config as an example to demonstrate how to collect systemd journal logs from the /logtail_host/var/log/journal directory on all nodes in the cluster.

You can specify existing projects and Logstores in the project and logstore fields to receive the data, or let the system generate them automatically.

apiVersion: log.alibabacloud.com/v1alpha1
kind: AliyunLogConfig
metadata:
  # The resource name must be unique within the current ACK cluster.
  name: systemd-journal-log-config
  namespace: kube-system
spec:
  # Specify the name of the project to which the collected data is reported. Default value is k8s-log-<your_cluster_id>. Custom names are also supported. If the specified Project does not exist, the system will automatically create it.
  project: k8s-log-<your_cluster_id>
  # Specify the name of the Logstore to which the collected data is reported. If the specified Logstore does not exist, the system will automatically create it.
  logstore: systemd-journal-log
  # The Logtail configuration.
  logtailConfig:
    # Specify the type of the data source. To collect text logs, set the value to file.
    inputType: plugin
    # Specify the name of the Logtail collection configuration.
    configName: systemd-journal-log-config
    inputDetail:
      plugin:
        inputs:
        - detail:
            JournalPaths:
            - "/logtail_host/var/log/journal"
            Kernel: true
            ParsePriority: true
            ParseSyslogFacility: true
          type: service_journal

Step 2: Query and analyze collected data

After configuring the collection, log on to the Simple Log Service console to query the systemd journal logs from all nodes within the specified project and Logstore. For more information, see Query and analyze logs.

References

logtail-ds can collect systemd journal logs directly from binary files. For more information, see Collect systemd journal logs.