This topic describes how to install Logtail components in a Kubernetes cluster.
Background information
To collect container logs from a Kubernetes cluster, you must install Logtail components.- Create a ConfigMap named alibaba-log-configuration. The ConfigMap contains the configuration information of Log Service, such as projects.
- Optional. Create a Custom Resource Definition (CRD) named AliyunLogConfig.
- Optional. Create a Deployment controller named alibaba-log-controller. The Deployment controller is used to monitor the changes in the AliyunLogConfig CRD and the creation of Logtail configurations.
- Create a DaemonSet named logtail-ds to collect logs from nodes.
Alibaba Cloud Container Service for Kubernetes (ACK) clusters
You can install Logtail components in an existing ACK cluster. You can also install Logtail components when you create an ACK cluster. To install Logtail components when you create an ACK cluster, you must select Enable Log Service.
Install Logtail components in an existing ACK cluster
- If your ACK cluster is a dedicated Kubernetes cluster or a managed Kubernetes cluster, you can follow the instructions that are provided in this section to install Logtail components in your cluster.
For information about how to collect text logs and standard output (stdout) from containers in a serverless Kubernetes (ASK) cluster, see Use Log Service to collect application logs.
- If your ACK cluster and Log Service belong to different Alibaba Cloud accounts, you must configure the ID of the Alibaba Cloud account to which your Log Service belongs as a user identifier for your cluster. For more information, see Configure the ID of an Alibaba Cloud account as a user identifier.
- Log on to the ACK console.
- In the left-side navigation pane, click Clusters.
- On the Clusters page, find and click the cluster in which you want to install Logtail components.
- In the left-side navigation pane of the page that appears, choose .
- On the Logs and Monitoring tab, find logtail-ds and click Install. After logtail-ds is installed, Log Service automatically creates a project named
k8s-log -${your_k8s_cluster_id}
, a machine group namedk8s-group-${your_k8s_cluster_id}
, and a Logstore namedconfig-operation-log
in the project.Important Do not delete theconfig-operation-log
Logstore.
Install Logtail components when you create an ACK cluster
- Log on to the ACK console.
- In the left-side navigation pane, click Clusters.
- On the Clusters page, click Create Kubernetes Cluster.
- In the Component Configurations step, select Enable Log Service. Note In this example, only the steps that are required to activate Log Service are provided. For more information about how to create an ACK cluster, see Create an ACK managed cluster.
If you select Enable Log Service, the system prompts you to create a Log Service project. For more information about how logs are managed in Log Service, see Project. You can use one of the following methods to create a project:
- Select Project
You can select an existing project to manage the container logs that are collected.
- Create Project
Log Service automatically creates a project named
k8s-log-{ClusterID}
to manage the container logs that are collected.ClusterID
indicates the unique ID of the ACK cluster that is created.
After the Logtail components are installed, a machine group namedk8s-group-${your_k8s_cluster_id}
and a Logstore namedconfig-operation-log
are automatically created in your project.Important Do not delete theconfig-operation-log
Logstore. - Select Project
Self-managed Kubernetes clusters
- Log on to the Log Service console.
- Create a project whose name starts with
k8s-log-custom-
.Example: k8s-log-custom-sd89ehdq. For more information, see Create a project. - Log on to your Kubernetes cluster.
- Run the following commands to install Logtail and dependent components. Important
- Make sure that the kubectl command-line tool is installed on the machine on which you want to run the commands.
- alibaba-log-controller is available only in Kubernetes 1.6 or later.
- If you no longer need to use CRDs, you can delete the alibaba-cloud-log/templates/alicloud-log-config.yaml file and rerun the following commands. If the
./alicloud-log-k8s-custom-install.sh: line 111: /root/alibaba-cloud-log/templates/alicloud-log-crd.yaml: No such file or directory
error message appears, you can ignore the error.
After Logtail and the components are installed, a machine group named
k8s-group-${your_k8s_cluster_id}
and a Logstore namedconfig-operation-log
are automatically created in your project.Important- Do not delete the
config-operation-log
Logstore. - If you install Logtail components in a self-managed Kubernetes cluster, Logtail is granted the
privileged
permissions. This helps prevent thecontainer text file busy
error that occurs when other pods are deleted. For more information, see Bug 1468249, Bug 1441737, and Issue 34538.
Error
- How do I view the version of a container image? You can view the version of a container image by visiting one of the following image repository:
- How do I collect container logs from multiple Kubernetes clusters to the same Log Service project?
- Alibaba Cloud ACK clusters
If you want to collect container logs from multiple ACK clusters to the same Log Service project, you must select the same project when you create the ACK clusters.
- Self-managed Kubernetes clusters
If you want to collect container logs from multiple self-managed Kubernetes clusters to the same Log Service project, you must set the {your-project-suffix} parameter to the same value that you specified when you install Logtail components in each Kubernetes cluster.
Note You can collect container logs from multiple self-managed Kubernetes clusters to the same Log Service project only if the Kubernetes clusters reside in the same region. - Alibaba Cloud ACK clusters
- How do I view the logs of Logtail?
The logs of Logtail are stored in the ilogtail.LOG and logtail_plugin.LOG files in the /usr/local/ilogtail/ directory of a Logtail container.
The standard output of the container is irrelevant to this case. Ignore the following standard output:
start umount useless mount points, /shm$|/merged$|/mqueue$ umount: /logtail_host/var/lib/docker/overlay2/3fd0043af174cb0273c3c7869500fbe2bdb95d13b1e110172ef57fe840c82155/merged: must be superuser to unmount umount: /logtail_host/var/lib/docker/overlay2/d5b10aa19399992755de1f85d25009528daa749c1bf8c16edff44beab6e69718/merged: must be superuser to unmount umount: /logtail_host/var/lib/docker/overlay2/5c3125daddacedec29df72ad0c52fac800cd56c6e880dc4e8a640b1e16c22dbe/merged: must be superuser to unmount ...... xargs: umount: exited with status 255; aborting umount done start logtail ilogtail is running logtail status: ilogtail is running
- How do I view the status of the Logtail component in Kubernetes clusters? Run the following commands:
kubectl get deploy alibaba-log-controller -n kube-system kubectl get ds logtail-ds -n kube-system
- What do I do if alibaba-log-controller fails to start? Check whether alibaba-log-controller is installed by using the following method:
- Run the installation command on the master node of your Kubernetes cluster.
- Specify the ID of your Kubernetes cluster in the installation command.
If alibaba-log-controller is not installed by using the preceding method, run the
kubectl delete -f deploy
command to delete the installation template that is generated. Then, rerun the installation command. - How do I view the status of Logtail DaemonSet in a Kubernetes cluster? Run the
kubectl get ds -n kube-system
command to view the status of Logtail DaemonSet.Note The default namespace in which a Logtail container resides is kube-system. - How do I view the version number, IP address, startup time, and status of Logtail?
- Run the following command to view the status of Logtail:
kubectl get po -n kube-system | grep logtail
The following output is returned:
NAME READY STATUS RESTARTS AGE logtail-ds-gb92k 1/1 Running 0 2h logtail-ds-wm7lw 1/1 Running 0 4d
- Run the following command to view the version number and IP address of Logtail:
kubectl exec logtail-ds-gb92k -n kube-system cat /usr/local/ilogtail/app_info.json
The following output is returned:
{ "UUID" : "", "hostname" : "logtail-ds-gb92k", "instance_id" : "0EBB2B0E-0A3B-11E8-B0CE-0A58AC140402_172.20.4.2_1517810940", "ip" : "192.0.2.0", "logtail_version" : "0.16.2", "os" : "Linux; 3.10.0-693.2.2.el7.x86_64; #1 SMP Tue Sep 12 22:26:13 UTC 2017; x86_64", "update_time" : "2021-02-05 06:09:01" }
- Run the following command to view the status of Logtail:
- How do I view the operational logs of Logtail? The operational logs of Logtail are stored in the ilogtail.LOG file in the /usr/local/ilogtail/ directory. If the log file is rotated, the generated files are compressed and stored as ilogtail.LOG.x.gz. Run the following command to view logs:
kubectl exec logtail-ds-gb92k -n kube-system tail /usr/local/ilogtail/ilogtail.LOG
The following output is returned:[2018-02-05 06:09:02.168693] [INFO] [9] [build/release64/sls/ilogtail/LogtailPlugin.cpp:104] logtail plugin Resume:start [2018-02-05 06:09:02.168807] [INFO] [9] [build/release64/sls/ilogtail/LogtailPlugin.cpp:106] logtail plugin Resume:success [2018-02-05 06:09:02.168822] [INFO] [9] [build/release64/sls/ilogtail/EventDispatcher.cpp:369] start add existed check point events, size:0 [2018-02-05 06:09:02.168827] [INFO] [9] [build/release64/sls/ilogtail/EventDispatcher.cpp:511] add existed check point events, size:0 cache size:0 event size:0 success count:0
- How do I restart Logtail for a pod?
- Stop Logtail.
In the following command,
logtail-ds-gb92k -n
specifies the container, andkube-system
specifies the namespace. Configure the parameters based on your business scenario:kubectl exec logtail-ds-gb92k -n kube-system /etc/init.d/ilogtaild stop
If the following output is returned, Logtail is stopped:kill process Name: ilogtail pid: 7 kill process Name: ilogtail pid: 9 stop success
- Start Logtail.
In the following command,
logtail-ds-gb92k -n
specifies the container, andkube-system
specifies the namespace. Configure the parameters based on your business scenario:kubectl exec logtail-ds-gb92k -n kube-system /etc/init.d/ilogtaild start
If the following output is returned, Logtail is started:ilogtail is running
- Stop Logtail.
- How do I collect the logs of control plane components?
- To collect the logs of control plane components from an ACK managed cluster, you can enable the log collection feature in the ACK console. For more information, see Collect the logs of control plane components in ACK Pro clusters.
- For information about how to collect the logs of control plane components from a self-managed Kubernetes cluster or an ACK dedicated cluster, see Use the Log Service console to collect container stdout and stderr in DaemonSet mode.
What to do next
Create Logtail configurations to collect container logs.- DaemonSet
- For information about how to collect container logs by using CRDs, see Use CRDs to collect container logs in DaemonSet mode.
- For information about how to collect container stdout and stderr logs by using the Log Service console, see Use the Log Service console to collect container stdout and stderr in DaemonSet mode.
- For information about how to collect container text logs by using the Log Service console, see Collect log data from containers by using Log Service.
- Sidecar
- For information about how to collect container logs by using CRDs, see Use CRDs to collect container text logs in Sidecar mode.
- For information about how to collect container logs by using the Log Service console, see Use the Log Service console to collect container text logs in Sidecar mode.