Container Service for Kubernetes is integrated with Log Service. When you create a cluster, you can enable Log Service to collect container logs, including standard outputs and text files.

Create a Kubernetes cluster

To create a Kubernetes cluster, use the following steps:

  1. Log on to the Container Service console.
  2. In the left-side navigation pane, choose Clusters > Clusters.
  3. In the upper-right corner, click Create Kubernetes Cluster. For more information about the configurations, see Create a cluster.
  4. In the Component Configuration section, select the Enable Log Service check box to install the logging plug-in.
  5. After you select the check box, you need to specify a project to manage log data. For more information about projects, see Projects.
    • You can select an existing project.
      Collect logs 1
    • You can also choose to create a new project. The project is named k8s-log-{ClusterID} by default, where ClusterID is the unique identifier of the cluster.
      Collect logs 2
  6. After you set the other parameters, click Create Cluster in the upper-right corner. In the dialog box that appears, click OK to create the cluster.
    You can find the newly created cluster on the Clusters page.
    Create a cluster

Install Log Service component logtail

If you have already created a Kubernetes cluster, take the following steps to start using Log Service:

  1. Log on to the Container Service console.
  2. In the left-side navigation pane, choose Clusters > Clusters. The Clusters page appears.
  3. Select the target cluster and click Manage in the Actions column. The Basic Information page appears.
    Clusters page
  4. In the left-side navigation pane, click Components. In the Optional Components section, find component logtail-ds.
  5. On the right of logtail-ds, click Install.
    Install logtail

Upgrade Log Service component logtail

If your log service component is outdated, you can take the steps introduced in Install Log Service component logtail to find logtail-ds on the Components page. You can then click Upgrade on the right of logtail-ds to upgrade it.Upgrade logtail-ds

Configure Log Service when you create an application

When you create an application in the Container Service console, you can configure Log Service to collect container logs. You can use the console wizard or YAML templates to create applications.

Use the console wizard to create an application

  1. Log on to the Container Service console.
  2. In the left-side navigation pane, choose Applications > Deployments and then click Create from Image in the upper-right corner.
  3. Specify the application name, cluster, namespace, number of replicas, and application type. Then click Next to go to the Container page.
    Container page
  4. On the Container page, specify the image. In this example, NGINX is selected.
    The following text describes configurations about Log Service only. For more information about other configuration items, see Create deployments by using images.
  5. In the Log section, click the plus sign (+) to create collection configuration entries, each of which consists of a Logstore name and a log path.
    • Logstore name: The name of the Logstore that stores log data. If the specified Logstore does not exist, the system automatically creates the Logstore under the project that is associated with the cluster.
      Note The Logstore name cannot contain underscores (_). Use hyphens (-) instead.
    • Log path: The path of the log that you want to collect. For example, /usr/local/tomcat/logs/catalina.*.log indicates Tomcat log files.
      Note If you set the log path to stdout, standard outputs and error messages are collected.

      For each configuration entry, an environment variable is automatically created for the corresponding Logstore. By default, the simple mode is used to collect logs. To enable other collection modes, go to the Log Service console, and select the corresponding project (prefixed with k8s-log by default) and Logstore to change configurations.

    Configure Logstore
  6. Custom tags.
    Click the plus sign (+) to create custom tags, each of which is a key-value pair that will be appended to collected logs. You can use custom tags to add marks to log data. For example, you can use a tag to denote the log version.
    Custom tags
  7. After you set the other parameters, click Next to set advanced settings. For more information about advanced settings, see Create deployments by using images.

Use a YAML template to create an application

  1. In the left-side navigation pane, choose Applications > Deployments. On the Deployments page that appears, click Create from Template in the upper-right corner.
  2. YAML templates follow the Kubernetes syntax. You can use the ENV field to set collection configurations and custom tags. You also need to create volumeMounts and volumes. A simple example is as follows:
    apiVersion: v1
    kind: Pod
    metadata:
      name: my-demo
    spec:
      containers:
      - name: my-demo-app
        image: 'registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest'
        env:
        ######### Configure environment variables ###########
        - name: aliyun_logs_log-stdout
          value: stdout
        - name: aliyun_logs_log-varlog
          value: /var/log/*.log
        - name: aliyun_logs_mytag1_tags
          value: tag1=v1
        ###############################
        ######### Configure volume mounting ###########
        volumeMounts:
        - name: volumn-sls-mydemo
          mountPath: /var/log
      volumes:
      - name: volumn-sls-mydemo
        emptyDir: {}
      ###############################
    • Specify the following configurations in order based on your needs:
    • Use environment variables to specify collection configurations and custom tags. All environment variables must have the aliyun_logs_ prefix.
    • Collection configurations must be in the following format:
      - name: aliyun_logs_{Logstore name}
        value: {Log path}                            

      In the preceding example, two environment variables are used to set collection configurations. Environment variable aliyun_logs_log-stdout instructs the system to create a Logstore named log-stdout, which collects standard outputs of the container.

      Note A Logstore name cannot contain underscores (_). You can use hyphens (-) instead.
    • Custom tags must be in the following format:
      - name: aliyun_logs_{Tag name without underscores (_)}_tags
        value: {Tag name}={Tag value}                            
      After a custom tag is set, it is automatically appended to certain log fields when logs from the specified container are collected.
    • If you specify a log path other than the standard output, you need to add the volumeMounts field.

      In the preceding example, the volumeMounts field is added and its value is /var/log. This indicates that /var/log/*.log files are collected.

  3. After you edit the YAML template, click Create to submit the configurations.

Advanced configurations

Environment variables support advanced configurations, which you can use to customize log collection. The following table provides details about these variables.
Variable Description Example Note
aliyun_logs_{key}
  • Required. The key can contain lowercase letters, digits, and hyphens (-), and cannot contain underscores (_).
  • If environment variable aliyun_logs_{key}_logstore is not set, a Logstore named {key} is created to store logs.
  • To collect standard outputs of the container, set the value to stdout. You can also set the value to other log paths.
  • - name: aliyun_logs_catalina
    
       stdout
  • - name: aliyun_logs_access-log
    
       /var/log/nginx/access.log
aliyun_logs_{key}_tags Optional. This variable is used to add marks to log data. It must be in the format of {tag-key}={tag-value}.
- name: aliyun_logs_catalina_tags

   app=catalina
-
aliyun_logs_{key}_project Optional. This variable specifies a project in Log Service. Default is the project that you specified when you create the cluster.
- name: aliyun_logs_catalina_project

   my-k8s-project
The region of the project must be the same as where your Logtail is located.
aliyun_logs_{key}_logstore Optional. This variable specifies a Logstore in Log Service. Default is the Logstore named after {key}.
- name: aliyun_logs_catalina_tags

   my-logstore
-
aliyun_logs_{key}_shard Optional. This variable specifies the number of shards in the Logstore. Valid values: 1 to 10. Default is 2.
- name: aliyun_logs_catalina_shard

   4
-
aliyun_logs_{key}_ttl Optional. This variable specifies the number of days for which log data is retained. Valid values: 1 to 3650.
  • To retain log data permanently, set the value to 3650.
  • Default is 90.
- name: aliyun_logs_catalina_ttl

   3650
-
aliyun_logs_{key}_machinegroup Optional. This variable specifies the machine group of the application. Default is the machine group where your Logtail is located.
- name: aliyun_logs_catalina_machinegroup

   my-machine-group
-
  • Scenario 1: Collect logs from multiple applications and store them in the same Logstore

    To meet the needs of this scenario, use the aliyun_logs_{key}_logstore variable. The following example collects standard outputs from two applications and store them in stdout-logstore.

    The environment variables of Application 1 are as follows:
    ######### Configure environment variables ###########
        - name: aliyun_logs_app1-stdout
          value: stdout
        - name: aliyun_logs_app1-stdout_logstore
          value: stdout-logstore
    The environment variables of Application 2 are as follows:
    ######### Configure environment variables ###########
        - name: aliyun_logs_app2-stdout
          value: stdout
        - name: aliyun_logs_app2-stdout_logstore
          value: stdout-logstore
  • Scenario 2: Collect logs from different applications and store them separately in different projects
    To meet the needs of this scenario, use the following steps:
    1. Create a machine group in each project and set the machine group ID in the following format: k8s-group-{cluster-id}, where {cluster-id} is the ID of the cluster. You can customize machine group names.
    2. Specify information about the project, Logstore, and machine group in the environment variables of each application.
      ######### Configure environment variables ###########
          - name: aliyun_logs_app1-stdout
            value: stdout
          - name: aliyun_logs_app1-stdout_project
            value: app1-project
          - name: aliyun_logs_app1-stdout_logstore
            value: app1-logstore
          - name: aliyun_logs_app1-stdout_machinegroup
            value: app1-machine-group

View log data

The following example shows how to view logs of the Tomcat application created through the console wizard. Tomcat application logs are stored in Log Service. Take the following steps to view log data:

  1. Log on to the Log Service console.
  2. Select the target project. By default, the project ID is k8s-log-{Cluster ID}.
  3. In the Logstores list, find the target Logstore and click Search in the Log Search column.
    Query
  4. In this example, you can view standard outputs of the Tomcat application and container logs. You can also find custom tags that are appended to certain log fields.
    Log field

Additional information

  1. By default, the simple mode is used to collect logs. Logs are collected line by line and not parsed. For more information about advanced configurations, see the following topics:
  2. In addition to the console, you can directly use CRDs to collect Kubernetes logs. .
  3. For more information about troubleshooting, see Troubleshoot collection errors.