This topic describes how to install the Logtail component in a Kubernetes cluster.

Background information

Before you can collect container logs from a Kubernetes cluster, you must install the Logtail component.
When you install the Logtail component, the following operations are automatically completed:
  1. A Custom Resource Definition (CRD) named aliyunlogconfigs is created.
  2. The alibaba-log-controller workload is deployed.
  3. Logtail is installed in DaemonSet mode.

Alibaba Cloud Container Service for Kubernetes (ACK) clusters

You can install the Logtail component in an existing ACK cluster. You can also install the Logtail component when you create an ACK cluster. To install the Logtail component when you create an ACK cluster, you must select Enable Log Service.

Install the Logtail component in an existing ACK cluster

Notice If your ACK cluster is a dedicated Kubernetes cluster or a managed Kubernetes cluster, you can follow the instructions in this section to install the Logtail component in your ACK cluster.
  1. Log on to the ACK console.
  2. In the left-side navigation pane, click Clusters.
  3. On the Clusters page, find and click the cluster in which you want to install the Logtail component.
  4. In the left-side navigation pane of the page that appears, choose Operations > Add-ons.
  5. On the Logs and Monitoring tab, find the logtail-ds component and click Install.
    After the component is installed, a machine group named k8s-group-${your_k8s_cluster_id} and a Logstore named config-operation-log are automatically created in the project that you use.
    Notice Do not delete the config-operation-log Logstore.

Install the Logtail component when you create an ACK cluster

  1. Log on to the ACK console.
  2. In the left-side navigation pane, click Clusters.
  3. On the Clusters page, click Create Cluster.
  4. In the Component Configurations step, select Enable Log Service.
    Note In this example, only the steps to enable Log Service are described. For more information about how to create an ACK cluster, see Create an ACK managed cluster.

    If you select Enable Log Service, the system prompts you to create a Log Service project. For more information about how logs are managed in Log Service, see Project. You can use one of the following methods to create a project:

    • Select Project

      You can select an existing project to manage the container logs that are collected.

      Install the Logtail component
    • Create Project

      Log Service automatically creates a project named k8s-log-{ClusterID} to manage the container logs that are collected. ClusterID indicates the unique ID of the ACK cluster that you create.

      Install the Logtail component
    After the component is installed, a machine group named k8s-group-${your_k8s_cluster_id} and a Logstore named config-operation-log are automatically created in the project that you use.
    Notice Do not delete the config-operation-log Logstore.

Self-managed Kubernetes clusters

  1. Log on to the Log Service console.
  2. Create a project whose name starts with k8s-log-custom-.
    Example: k8s-log-custom-sd89ehdq. For more information, see Create a project.
  3. Log on to your Kubernetes cluster.
  4. Run the following commands to install Logtail and dependent components.
    Notice
    • Make sure that the kubectl command-line tool is installed on the machine on which you want to run the commands.
    • The alibaba-log-controller workload is available only in Kubernetes 1.6 or later.
    • If you no longer need to use CRDs, you can delete the alibaba-cloud-log/templates/alicloud-log-config.yaml file and run the following commands again. If the ./alicloud-log-k8s-custom-install.sh: line 111: /root/alibaba-cloud-log/templates/alicloud-log-crd.yaml: No such file or directory error message appears, you can ignore the error.
    1. Download the installation script. 
      wget http://logtail-release-cn-hangzhou.oss-cn-hangzhou.aliyuncs.com/kubernetes/alicloud-log-k8s-custom-install.sh
    2. Modify permissions to limit the access to the installation script. 
      chmod 744 ./alicloud-log-k8s-custom-install.sh
    3. Install Logtail and dependent components. 
      sh ./alicloud-log-k8s-custom-install.sh your-project-suffix region-id aliuid access-key-id access-key-secret

      The following table describes the parameters that are included in the preceding command. You can configure the parameters based on your business requirements.

      Parameter Description
      your-project-suffix The part that follows k8s-log-custom- in the name of your project. Use the project that you created in Step 2. For example, if the project name is k8s-log-custom-sd89ehdq, set the value to sd89ehdq.
      region-id The ID of the region where your project resides. For example, the ID of the China (Hangzhou) region is cn-hangzhou. For more information, see Supported regions.
      aliuid The ID of your Alibaba Cloud account. For more information, see Step 1: Obtain the ID of the Alibaba Cloud account for which Log Service is activated.
      access-key-id The AccessKey ID of your Alibaba Cloud account. We recommend that you use the AccessKey pair of a RAM user and attach the AliyunLogFullAccess policy to the RAM user. For more information, see Create a RAM user and authorize the RAM user to access Log Service.
      access-key-secret The AccessKey secret of your Alibaba Cloud account. We recommend that you use the AccessKey pair of a RAM user and attach the AliyunLogFullAccess policy to the RAM user. For more information, see Create a RAM user and authorize the RAM user to access Log Service.

    After the components are installed, a machine group named k8s-group-${your_k8s_cluster_id} and a Logstore named config-operation-log are automatically created in the project that you use.

    Notice
    • Do not delete the config-operation-log Logstore.
    • If you install the components in a self-managed Kubernetes cluster, Logtail is granted the privileged permissions by default. This prevents the container text file busy error that may occur when other pods are deleted. For more information, see Bug 1468249, Bug 1441737, and Issue 34538.

FAQ

  • How do I collect container logs from multiple Kubernetes clusters to the same Log Service project?
    • Alibaba Cloud ACK clusters

      If you want to collect container logs from multiple ACK clusters to the same Log Service project, you must select the same project when you create the ACK clusters.

    • Self-managed Kubernetes clusters

      If you want to collect container logs from multiple self-managed Kubernetes clusters to the same Log Service project, you must set the {your-project-suffix} parameter to the same value when you install the Logtail component in each of the Kubernetes clusters.

    Note You can collect container logs from multiple self-managed Kubernetes clusters to the same Log Service project only if the Kubernetes clusters reside in the same region.
  • How do I view the logs of Logtail?

    The logs of Logtail are stored in the files named ilogtail.LOG and logtail_plugin.LOG in the /usr/local/ilogtail/ directory of a Logtail container.

    The stdout and stderr of the Logtail container are not for reference. You can ignore the following stdout and stderr: 

    start umount useless mount points, /shm$/merged$/mqueue$
umount: /logtail_host/var/lib/docker/overlay2/3fd0043af174cb0273c3c7869500fbe2bdb95d13b1e110172ef57fe840c82155/merged: must be superuser to unmount
umount: /logtail_host/var/lib/docker/overlay2/d5b10aa19399992755de1f85d25009528daa749c1bf8c16edff44beab6e69718/merged: must be superuser to unmount
umount: /logtail_host/var/lib/docker/overlay2/5c3125daddacedec29df72ad0c52fac800cd56c6e880dc4e8a640b1e16c22dbe/merged: must be superuser to unmount
......
xargs: umount: exited with status 255; aborting
umount done
start logtail
ilogtail is running
logtail status:
ilogtail is running
  • How do I view the status of the Logtail component in Kubernetes clusters?
    Run the following commands: 
    kubectl get deploy alibaba-log-controller -n kube-system
kubectl get ds logtail-ds -n kube-system
  • What do I do if alibaba-log-controller fails to start?
    Check whether alibaba-log-controller is installed by using the following method:
    • Run the installation command on the master node of your Kubernetes cluster.
    • Specify the ID of your Kubernetes cluster in the installation command.

    If alibaba-log-controller is not installed by using the preceding method, run the kubectl delete -f deploy command to delete the installation template that is generated. Then, run the installation command again.

  • How do I view the status of the Logtail DaemonSet in a Kubernetes cluster?
    Run the kubectl get ds -n kube-system command to view the status of the Logtail DaemonSet.
    Note The default namespace to which a Logtail container belongs is kube-system.
  • How do I view the version number, IP address, startup time, and status of Logtail?
    • Run the following command to view the status of Logtail: 
      kubectl get po -n kube-system  grep logtail

      The following result is returned:

      NAME            READY     STATUS    RESTARTS   AGE
logtail-ds-gb92k   1/1       Running   0          2h
logtail-ds-wm7lw   1/1       Running   0          4d
    • Run the following command to view the version number and IP address of Logtail: 
      kubectl exec logtail-ds-gb92k -n kube-system cat /usr/local/ilogtail/app_info.json

      The following result is returned:

      {
   "UUID" : "",
   "hostname" : "logtail-ds-gb92k",
   "instance_id" : "0EBB2B0E-0A3B-11E8-B0CE-0A58AC140402_172.20.4.2_1517810940",
   "ip" : "192.0.2.0",
   "logtail_version" : "0.16.2",
   "os" : "Linux; 3.10.0-693.2.2.el7.x86_64; #1 SMP Tue Sep 12 22:26:13 UTC 2017; x86_64",
   "update_time" : "2021-02-05 06:09:01"
}
  • How do I view the run logs of Logtail?

    The run logs of Logtail are stored in the ilogtail.LOG file in the /usr/local/ilogtail/ directory. If the log file is rotated, the generated files are compressed and stored as ilogtail.LOG.x.gz.

    Example: 
    [root@iZbp1dsu6v77zfb40qfbiaZ ~]# kubectl exec logtail-ds-gb92k -n kube-system tail /usr/local/ilogtail/ilogtail.LOG
[2018-02-05 06:09:02.168693] [INFO] [9] [build/release64/sls/ilogtail/LogtailPlugin.cpp:104] logtail plugin Resume:start
[2018-02-05 06:09:02.168807] [INFO] [9] [build/release64/sls/ilogtail/LogtailPlugin.cpp:106] logtail plugin Resume:success
[2018-02-05 06:09:02.168822] [INFO] [9] [build/release64/sls/ilogtail/EventDispatcher.cpp:369] start add existed check point events, size:0
[2018-02-05 06:09:02.168827] [INFO] [9] [build/release64/sls/ilogtail/EventDispatcher.cpp:511] add existed check point events, size:0 cache size:0 event size:0 success count:0
  • How do I restart Logtail for a pod?
    1. Stop Logtail.

      In the following command, logtail-ds-gb92k -n specifies the container, and kube-system specifies the namespace. Configure the parameters based on your business requirements. 

      kubectl exec logtail-ds-gb92k -n kube-system /etc/init.d/ilogtaild stop
      If the following output is returned, Logtail is stopped. 
      kill process Name: ilogtail pid: 7
kill process Name: ilogtail pid: 9
stop success
    2. Start Logtail.

      In the following command, logtail-ds-gb92k -n specifies the container, and kube-system specifies the namespace. Configure the parameters based on your business requirements. 

      kubectl exec logtail-ds-gb92k -n kube-system /etc/init.d/ilogtaild start
      If the following output is returned, Logtail is started. 
      ilogtail is running

What to do next

Create Logtail configurations to collect container logs.