Container Service for Kubernetes (ACK) is integrated with Log Service. When you create an ACK cluster, you can enable Log Service by installing the logtail-ds component. After Log Service is enabled, it collects log data from containers of the ACK cluster. The log data includes the standard output and text files. This topic describes how to collect log data from containers by using Log Service.
Table of contents
Step 1: Install Logtail
When you create a cluster, select Enable Log service to install Logtail. You can also install Logtail for an existing cluster.
Install Logtail when you create a cluster:
- Log on to the ACK console.
- In the left-side navigation pane of the ACK console, click Clusters.
In the upper-right corner of the Clusters page, click Create Kubernetes Cluster.
In this example, only the steps to enable Log Service are described. For more information about how to create an ACK cluster, see Create an ACK managed cluster.
On the Component Configurations wizard page, select Enable Log Service to install Logtail in the cluster.
If you select the Enable Log Service check box, the system prompts you to create a Log Service project. For more information about the structure of the logs managed by Log Service, see Projects. You can use one of the following methods to create a Log Service project:
Click Select Project and select an existing project to manage the collected log.
Click Create Project. Then, a project named
k8s-log-{ClusterID}
is automatically created to manage the collected log. ClusterID indicates the unique ID of the cluster to be created.
After you set the parameters, click Create Cluster in the lower-right corner. In the message that appears, click OK.
After the ACK cluster is created, you can find the cluster that has Logtail enabled on the Clusters page.
Install Logtail for an existing cluster
- Log on to the ACK console.
- In the left-side navigation pane of the ACK console, click Clusters.
- On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
In the left-side navigation pane of the cluster details page, choose Operations > Add-ons. In the Logs and Monitoring section, find logtail-ds.
Click Install in the lower-right part of the logtail-ds card. In the Install logtail-ds dialog box, click OK.
If the logtail-ds version that you have already installed is not update-to-date, you can click Update in the lower-right part of the logtail-ds card to update the component.
After you update logtail-ds, the parameters of logtail-ds are reset. The settings and environment variables of logtail-ds or alibaba-log-controller will be overwritten. If you have customized the settings and environment variables, you need to reconfigure them later. For more information, see Manual upgrade.
Step 2: Configure Log Service when you create an application
You can configure Log Service to collect log data from containers when you create an application. To do this, you can use the ACK console or use a YAML file.
Use the wizard in the ACK console to create an application and configure Log Service
Log on to the ACK console and click Clusters in the left-side navigation pane.
On the Clusters page, click the name of a cluster and choose in the left-side navigation pane.
On the Deployments page, select a namespace from the Namespace drop-down list. Then, click Create from Image in the upper-right corner of the page.
On the Basic Information wizard page, specify Name, Replicas, and Type. Then, click Next to go to the Container wizard page.
Only parameters related to Log Service are described in the following section. For more information about other application parameters, see Create a Deployment.
In the Log Service section, configure the relevant parameters.
Configure Collection Configuration.
Click the plus sign (+) to add a configuration entry. Each configuration entry consists of Logstore and Log Path in Container (can be set to stdout).
Logstore: Specify the name of the Logstore that is used to store the collected log data. If the Logstore does not exist, ACK automatically creates a Logstore in the Log Service project that is associated with your ACK cluster.
NoteThe default log retention period of Logstores is 180 days.
Log Path in Container (Can be set to stdout): the path from which you want to collect log data. A value of /usr/local/tomcat/logs/catalina.*.log indicates that the log files of a Tomcat application are collected.
NoteWhen you set the value to stdout, stdout and stderr are collected.
All settings are added as configuration entries to the corresponding Logstore. By default, logs are collected in simple mode (by row). If you want to use other methods to collect log data, see Use the Log Service console to collect container text logs in DaemonSet mode and Use the Log Service console to collect container stdout and stderr in DaemonSet mode.
Set Custom Tag.
Click the plus sign (+) to add custom tags. Each tag is a key-value pair that is appended to the collected log data. You can use custom tags to mark log data. For example, you can use a tag to denote the application version.
After you configure the parameters, click Next to configure advanced settings.
For more information about the subsequent steps, , see Create a Deployment.
Use a YAML template to create an application and configure Log Service
Log on to the ACK console and click Clusters in the left-side navigation pane.
On the Clusters page, click the name of a cluster and choose in the left-side navigation pane.
On the Deployments page, select a namespace from the Namespace drop-down list. Then, click Create from YAML in the upper-right corner of the page.
Configure the YAML template.
YAML templates comply with the Kubernetes syntax. You can use
env
to define log collection configurations and custom tags. You must also set thevolumeMounts
andvolumes
parameters. The following code is an example:apiVersion: v1 kind: Pod metadata: name: my-demo spec: containers: - name: my-demo-app image: 'registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest' env: ######### Specify environment variables. ########### - name: aliyun_logs_log-stdout value: stdout - name: aliyun_logs_log-varlog value: /var/log/*.log - name: aliyun_logs_mytag1_tags value: tag1=v1 ############################### ######### Specify volume mounting. ########### volumeMounts: - name: volumn-sls-mydemo mountPath: /var/log volumes: - name: volumn-sls-mydemo emptyDir: {} ###############################
Perform the following steps in sequence based on your requirements:
NoteIf you have other log collection requirements, see Step 3: Configure advanced settings in the env field.
Add log collection configurations and custom tags by using environment variables. All environment variables related to log collection must use
aliyun_logs_
as the prefix.Add log collection configurations in the following format:
- name: aliyun_logs_log-stdout value: stdout - name: aliyun_logs_log-varlog value: /var/log/*.log
In the preceding example, two environment variables in the following format are added to the log collection configuration:
aliyun_logs_{key}
. The{keys}
of the environment variables arelog-stdout
andlog-varlog
.Environment variable
aliyun_logs_log-stdout
indicates that aLogstore
namedlog-stdout
is created to store thestdout
files collected from containers. The name of the collection configuration islog-stdout
. This way, the stdout of containers is collected to theLogstore
namedlog-stdout
.Environment variable
aliyun_logs_log-varlog
indicates that aLogstore
namedlog-varlog
is created to store the collected log files to the /var/log/*.log path. The name of the collection configuration islog-varlog
. This way, text files of the /var/log/*.log file are collected to theLogstore
namedlog-varlog
.
Add custom tags in the following format:
- name: aliyun_logs_mytag1_tags value: tag1=v1
After a tag is added, the tag is automatically appended to the log data that is collected from the container.
mytag1
specifies thetag name without underscores (_)
.
If you specify a log path to collect log files other than stdout, you must set the
volumeMounts
parameter.In the preceding YAML template, the mountPath field in
volumeMounts
is set to /var/log. This allows Logtail to collect log data from the /var/log/*.log file.
After you modify the YAML template, click Create to submit the configurations.
Step 3: Configure advanced settings in the env field
You can configure container environment variables to customize log collection. You can configure advanced settings to meet your log collection requirement.
You cannot use environment variables to configure log collection in edge computing scenarios.
Variable | Description | Example | Usage notes |
aliyun_logs_{key} |
|
|
|
aliyun_logs_{key}_tags | Optional. This variable is used to add tags to log data. The value must be in the following format: {tag-key}={tag-value}. |
| N/A |
aliyun_logs_{key}_project | Optional. The variable specifies a project in Log Service. By default, the project that you specified when you created the cluster is used. |
| The project must be deployed in the same region as Logtail. |
aliyun_logs_{key}_logstore | Optional. The variable specifies a Logstore in Log Service. By default, the Logstore is named {key}. |
| N/A |
aliyun_logs_{key}_shard | Optional. The variable specifies the number of shards of the Logstore. Valid values: 1 to 10. Default value: 2. Note If the Logstore that you specify already exists, this variable does not take effect. |
| N/A |
aliyun_logs_{key}_ttl | Optional. The variable specifies the log retention period. Valid values: 1 to 3650.
Note If the Logstore that you specify already exists, this variable does not take effect. |
| N/A |
aliyun_logs_{key}_machinegroup | Optional. This variable specifies the node group in which the application is deployed. The default node group is the one in which Logtail is deployed. For more information about how to use the variable, see Scenario 2: Collect log data from different applications and store them in different projects. |
| N/A |
aliyun_logs_{key}_logstoremode | Optional. This variable specifies the type of Logstore. Default value: standard. Valid values: Note If the Logstore that you specify already exists, this variable does not take effect.
|
| To use this variable, make sure that the logtail-ds image version is 1.3.1 or later. |
Scenario 1: Collect log data from multiple applications and store them in the same Logstore
In this scenario, set the aliyun_logs_{key}_logstore variable. The following example shows how to collect stdout from two applications and store the output in stdout-logstore.
The
{key}
of Application 1 is set toapp1-stdout
. The{key}
of Application 2 is set toapp2-stdout
.Configure the following environment variables for Application 1:
######### Specify environment variables. ########### - name: aliyun_logs_app1-stdout value: stdout - name: aliyun_logs_app1-stdout_logstore value: stdout-logstore
Configure the following environment variables for Application 2:
######### Specify environment variables. ########### - name: aliyun_logs_app2-stdout value: stdout - name: aliyun_logs_app2-stdout_logstore value: stdout-logstore
Scenario 2: Collect log data from different applications and store them in different projects
In this scenario, perform the following steps:
Create a machine group in each project and set the machine group ID in the following format: k8s-group-{cluster-id}, where
{cluster-id}
is the ID of the cluster. You can customize the machine group name.Specify the project, Logstore, and the machine group in the environment variables for each application. The name of the machine group is the same as that of the machine group that you created in the previous step.
In the following example, the {key} of Application 1 is set to
app1-stdout
. The {key} of Application 2 is set toapp2-stdout
.If the two applications are deployed in the same ACK cluster, you can use the same machine group for the applications.
Configure the following environment variables for Application 1:
######### Specify environment variables. ########### - name: aliyun_logs_app1-stdout value: stdout - name: aliyun_logs_app1-stdout_project value: app1-project - name: aliyun_logs_app1-stdout_logstore value: app1-logstore - name: aliyun_logs_app1-stdout_machinegroup value: app1-machine-group
Configure the following environment variables for Application 2:
######### Specify environment variables for Application 2.########### - name: aliyun_logs_app2-stdout value: stdout - name: aliyun_logs_app2-stdout_project value: app2-project - name: aliyun_logs_app2-stdout_logstore value: app2-logstore - name: aliyun_logs_app2-stdout_machinegroup value: app1-machine-group
Step 4: View log data
The following example shows how to view the log data of a Tomcat application created by using the wizard in the ACK console. After you complete the configuration, the log data of the Tomcat application is collected and stored in Log Service. You can log on to the Log Service or ACK console to view the container logs.
View container logs in the Log Service console
Log on to the Log Service console.
In the Projects section, click the project that is associated with the Kubernetes cluster to go to the Logstores tab. By default, the project name is in the format of k8s-log-{Kubernetes cluster ID}.
In the Logstore list, find the Logstore that is specified when you configure log collection. Move the pointer over the Logstore name and click the
icon. Then, click Search & Analysis.
In this example, you can view the stdout of the Tomcat application and text log files of containers. You can also find that custom tags are appended to the collected log data.
View container logs in the ACK console
Log on to the ACK console and click Clusters in the left-side navigation pane.
On the Clusters page, click the name of a cluster and choose in the left-side navigation pane.
On the Log Center page, click the Application Logs tab and specify the filter conditions. Then, click Select Logstore to view the logs of containers.
What to do next
You can view the logs collected from containers in ACK clusters in the Log Service console.
By default, Log Service collects log data by line and does not parse the log data. If you want to change the log collection mode and parse the log data, modify the log collection configurations in the Log Service console. For more information, see the following topics:
You can also use CustomResourceDefinition (CRD) objects to collect log data from Kubernetes clusters.
For more information, see What do I do if errors occur when I use Logtail to collect logs?