This topic describes how to use CRDs to collect Kubernetes container logs in the Sidecar mode.

Prerequisites

The Helm package alibaba-log-controller is installed. For more information, see Install Logtail.

Overview

In the Sidecar mode, the Logtail container shares the log directory with application containers in a pod. Application containers write logs to the shared directory and the Logtail container monitors the changes of log files in the shared directory and collects logs. For more information, see Sidecar container with a logging agent and How Pods manage multiple Containers.

Step 1: Install Sidecar

The configuration template for log collection in the Sidecar mode is as follows:
apiVersion: batch/v1
kind: Job
metadata:
  name: nginx-log-sidecar-demo
  namespace: default
spec:
  template:
    metadata:
      name: nginx-log-sidecar-demo
    spec:
      restartPolicy: Never
      containers:
      - name: nginx-log-demo
        image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
        command: ["/bin/mock_log"]
        args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
        volumeMounts:
        - name: nginx-log
          mountPath: /var/log/nginx
      ##### logtail sidecar container
      - name: logtail
        # more info: https://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
        # This image is released for every region
        image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
        # when receive sigterm, logtail will delay 10 seconds and then stop
        command:
        - sh
        - -c
        - /usr/local/ilogtail/run_logtail.sh 10
        livenessProbe:
          exec:
            command:
            - /etc/init.d/ilogtaild
            - status
          initialDelaySeconds: 30
          periodSeconds: 30
        resources:
          limits:
            memory: 512Mi
          requests:
            cpu: 10m
            memory: 30Mi
        env:
          ##### base config
          # user id
          - name: "ALIYUN_LOGTAIL_USER_ID"
            value: "${your_aliyun_user_id}"
          # user defined id
          - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
            value: "${your_machine_group_user_defined_id}"
          # config file path in logtail's container
          - name: "ALIYUN_LOGTAIL_CONFIG"
            value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
          ##### env tags config
          - name: "ALIYUN_LOG_ENV_TAGS"
            value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
          - name: "_pod_name_"
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: "_pod_ip_"
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
          - name: "_namespace_"
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: "_node_name_"
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: "_node_ip_"
            valueFrom:
              fieldRef:
                fieldPath: status.hostIP
        volumeMounts:
        - name: nginx-log
          mountPath: /var/log/nginx
      ##### share this volume
      volumes:
      - name: nginx-log
        emptyDir: {}
  1. Log on to your Kubernetes cluster. For more information, see Access Kubernetes clusters by using SSH key pairs.
  2. Configure the following basic parameters for log collection.
    ##### base config
              # user id
              - name: "ALIYUN_LOGTAIL_USER_ID"
                value: "${your_aliyun_user_id}"
              # user defined id
              - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
                value: "${your_machine_group_user_defined_id}"
              # config file path in logtail's container
              - name: "ALIYUN_LOGTAIL_CONFIG"
                value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
    Parameter Description
    ${your_region_config} The ID of the region and the type of the network where your project resides. For more information, see Table 1.
    • If your project resides in the public network, the parameter value is formatted in region-internet, for example, cn-hangzhou-Internet.
    • If your project resides in an internal network, the parameter value is formatted in region, for example, cn-hangzhou.
    ${your_aliyun_user_id} The unique ID of your Alibaba Cloud account. For more information, see Configure an account ID for a server.
    ${your_machine_group_user_defined_id} The custom identifier of your server group. The identifier must be unique in the region where your project resides. For more information, see Create a custom ID-based machine group.
  3. Configure the mount path of logs.
    volumeMounts:
    - name: nginx-log
    mountPath: /var/log/nginx
    • The Logtail container and application containers in a pod must share the same directory.
    • We recommend that you mount an emptyDir volume on the containers.
  4. Configure the latency for stopping log collection
    Generally, the latency configured for stopping log collection is 10 seconds. This value indicates that the Logtail container stops collecting logs 10 seconds after it is ordered to stop collection. This configuration prevents incomplete data collection.
    command:        
    - sh        
    - -c        
    - /usr/local/ilogtail/run_logtail.sh 10

Step 2: Configure log collection

You can configure an AliyunLogConfig CRD to collect logs. The following script shows how to configure a CRD. To delete collection configurations, you need to delete the corresponding CRD.
apiVersion: log.alibabacloud.com/v1alpha1      ## The default setting, which you do not need to modify.
kind: AliyunLogConfig                          ## The default setting, which you do not need to modify.
metadata:
  name: simple-stdout-example                  ## The resource name, which must be unique in the cluster.
spec:
  project: k8s-my-project                      ## The name of a project. If you do not specify this parameter, logs are collected to the project configured when you installed Log Service components on the cluster.
  logstore: k8s-stdout                         ## The Logstore name. A Logstore is automatically created if the specified Logstore does not exist.
  machineGroups:- nginx-log-sidecar            ## The name of the server group. If you do not specify this parameter, logs are collected from the server group where Logtail DaemonSets reside.
  shardCount: 2                                ## The number of Logstore shards. Valid values: 1 to 10. Default value: 2. This parameter is optional.
  lifeCycle: 90                                ## The retention days of log data in the Logstore. Valid values: 1 to 7300. Default value: 90. The value 7300 indicates that log data is permanently stored in the Logstore. This parameter is optional.
  logtailConfig:                               ## The Logtail settings.
    inputType: plugin                          ## The type of the data source to be collected. Only text logs can be collected.
    configName: simple-stdout-example          ## The name of the Logtail configuration file. This name must be the same as the resource name.
    inputDetail:                               ## The detailed settings of Logtail. For more information, see the examples.
      ...

In the Sidecar mode, only text logs can be collected. The value of the dockerFile parameter must be set to false.

For more information about the logtailConfig parameter, see Logtail configuration.

After the configuration file is created, the Logtail container automatically collects and uploads logs to Log Service. You can log on to the Log Service console to view the logs.

Example

The following example shows how to use a CRD to collect logs in the Sidecar mode from a user-created Kubernetes cluster deployed in a local IDC. The project created in Log Service for log collection resides in the China (Hangzhou) region. Logs are collected over the Internet. The volume mounted on the /var/log/nginx directory of the nginx-log-demo and Logtail containers is an emptyDir volume named nginx-log.

  1. Install Sidecar.
    apiVersion: batch/v1
    kind: Job
    metadata:
      name: nginx-log-sidecar-demo
      namespace: default
    spec:
      template:
        metadata:
          name: nginx-log-sidecar-demo
        spec:
          restartPolicy: Never
          containers:
          - name: nginx-log-demo
            image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
            command: ["/bin/mock_log"]
            args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
            volumeMounts:
            - name: nginx-log
              mountPath: /var/log/nginx
          ##### logtail sidecar container
          - name: logtail
            # more info: ttps://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
            # This image is released for every region.
            image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
            # when receive sigterm, logtail will delay 10 seconds and then stop
            command:
            - sh
            - -c
            - /usr/local/ilogtail/run_logtail.sh 10
            livenessProbe:
              exec:
                command:
                - /etc/init.d/ilogtaild
                - status
              initialDelaySeconds: 30
              periodSeconds: 30
            env:
              ##### base config
              # user id
              - name: "ALIYUN_LOGTAIL_USER_ID"
                value: "xxxxxxxxxx"
              # user defined id
              - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
                value: "nginx-log-sidecar"
              # config file path in logtail's container
              - name: "ALIYUN_LOGTAIL_CONFIG"
                value: "/etc/ilogtail/conf/cn-hangzhou-internet/ilogtail_config.json"
              ##### env tags config
              - name: "ALIYUN_LOG_ENV_TAGS"
                value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
              - name: "_pod_name_"
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.name
              - name: "_pod_ip_"
                valueFrom:
                  fieldRef:
                    fieldPath: status.podIP
              - name: "_namespace_"
                valueFrom:
                  fieldRef:
                    fieldPath: metadata.namespace
              - name: "_node_name_"
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              - name: "_node_ip_"
                valueFrom:
                  fieldRef:
                    fieldPath: status.hostIP
            volumeMounts:
            - name: nginx-log
              mountPath: /var/log/nginx
          ##### share this volume
          volumes:
          - name: nginx-log
            emptyDir: {}
  2. Configure log collection.
    • Set the log path for access logs to be collected to /var/log/nginx/access.log and the destination Logstore to nginx-access.
      # config for access log
      apiVersion: log.alibabacloud.com/v1alpha1
      kind: AliyunLogConfig
      metadata:
        # your config name, must be unique in you k8s cluster
        name: nginx-log-access-example
      spec:
        # project name to upload log
        project: k8s-nginx-sidecar-demo
        # logstore name to upload log
        logstore: nginx-access
        # machine group list to apply config, should be same with your sidecar' [ALIYUN_LOGTAIL_USER_DEFINED_ID]
        machineGroups:
        - nginx-log-sidecar
        # logtail config detail
        logtailConfig:
          # log file's input type is 'file'
          inputType: file
          # logtail config name, should be same with [metadata.name]
          configName: nginx-log-access-example
          inputDetail:
            # Set logType to common_reg_log.
            logType: common_reg_log
            # Specify the log path.
            logPath: /var/log/nginx
            # Specify the names of log files to be collected. You can include wildcards in the parameter value, for example, log_*.log.
            filePattern: access.log
            # Set the dockerFile parameter to false in the Sidecar mode.
            dockerFile: false
            # Use a regular expression to match the first log entry to be collected. For single-line log entries, the value is set to .*
            logBeginRegex: '. *'
            # Use a regular expression to parse logs
            regex: '(\S+)\s(\S+)\s\S+\s\S+\s"(\S+)\s(\S+)\s+([^"]+)"\s+(\S+)\s(\S+)\s(\d+)\s(\d+)\s(\S+)\s"([^"]+)"\s. *'
            # Specify the keys extracted from logs
            key : ["time", "ip", "method", "url", "protocol", "latency", "payload", "status", "response-size",user-agent"]
      # config for error log
    • Set the log path for error logs to be collected to /var/log/nginx/error.log and the destination Logstore to nginx-error.
      # config for error log
      apiVersion: log.alibabacloud.com/v1alpha1
      kind: AliyunLogConfig
      metadata:
        # your config name, must be unique in you k8s cluster
        name: nginx-log-error-example
      spec:
        # project name to upload log
        project: k8s-nginx-sidecar-demo
        # logstore name to upload log
        logstore: nginx-error
        # machine group list to apply config, should be same with your sidecar' [ALIYUN_LOGTAIL_USER_DEFINED_ID]
        machineGroups:
        - nginx-log-sidecar
        # logtail config detail
        logtailConfig:
          # log file's input type is 'file'
          inputType: file
          # logtail config name, should be same with [metadata.name]
          configName: nginx-log-error-example
          inputDetail:
            # Set logType to common_reg_log.
            logType: common_reg_log
            # Specify the log path.
            logPath: /var/log/nginx
            # Specify the names of log files to be collected. You can include wildcards in the parameter value, for example, log_*.log.
            filePattern: error.log
            # Set the dockerFile parameter to false in the Sidecar mode.
            dockerFile: false