Use CRDs to collect Kubernetes container logs in Sidecar mode - Data Collection| Alibaba Cloud Documentation Center

This topic describes how to install Sidecar. This topic also describes how to use a custom resource definition (CRD) to create a Logtail configuration to collect Kubernetes container logs in Sidecar mode.

Prerequisites

The Helm package alibaba-log-controller is installed. For more information, see Install Logtail.

Implementation

In Sidecar mode, the Logtail container shares a log directory with an application container in a pod. The application container writes logs to the shared directory and the Logtail container monitors the changes of log files in the shared directory and collects logs. For more information, see Sidecar container with a logging agent and How Pods manage multiple containers.

Step 1: Install Sidecar

The following sample code shows how to install Sidecar:

apiVersion: batch/v1
kind: Job
metadata:
  name: nginx-log-sidecar-demo
  namespace: default
spec:
  template:
    metadata:
      name: nginx-log-sidecar-demo
    spec:
      restartPolicy: Never
      containers:
      - name: nginx-log-demo
        image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
        command: ["/bin/mock_log"]
        args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
        volumeMounts:
        - name: nginx-log
          mountPath: /var/log/nginx
      ##### logtail sidecar container
      - name: logtail
        # more info: https://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
        # this images is released for every region
        image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
        # when recevie sigterm, logtail will delay 10 seconds and then stop
        command:
        - sh
        - -c
        - /usr/local/ilogtail/run_logtail.sh 10
        livenessProbe:
          exec:
            command:
            - /etc/init.d/ilogtaild
            - status
          initialDelaySeconds: 30
          periodSeconds: 30
        resources:
          limits:
            memory: 512Mi
          requests:
            cpu: 10m
            memory: 30Mi
        env:
          ##### base config
          # user id
          - name: "ALIYUN_LOGTAIL_USER_ID"
            value: "${your_aliyun_user_id}"
          # user defined id
          - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
            value: "${your_machine_group_user_defined_id}"
          # config file path in logtail's container
          - name: "ALIYUN_LOGTAIL_CONFIG"
            value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"
          ##### env tags config
          - name: "ALIYUN_LOG_ENV_TAGS"
            value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
          - name: "_pod_name_"
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: "_pod_ip_"
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
          - name: "_namespace_"
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: "_node_name_"
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: "_node_ip_"
            valueFrom:
              fieldRef:
                fieldPath: status.hostIP
        volumeMounts:
        - name: nginx-log
          mountPath: /var/log/nginx
      ##### share this volume
      volumes:
      - name: nginx-log
        emptyDir: {}

Log on to your Kubernetes cluster.

Specify the following basic parameters for the Logtail configuration.

##### base config
          # user id
          - name: "ALIYUN_LOGTAIL_USER_ID"
            value: "${your_aliyun_user_id}"
          # user defined id
          - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
            value: "${your_machine_group_user_defined_id}"
          # config file path in logtail's container
          - name: "ALIYUN_LOGTAIL_CONFIG"
            value: "/etc/ilogtail/conf/${your_region_config}/ilogtail_config.json"


Parameter	Description
`${your_region_config}`	The ID of the region where your project resides and the type of the network where your project resides. For more information about regions, see Table 1. If your project is accessed over the Internet, set the value in the `region-internet` format, for example, cn-hangzhou-Internet. If your project is accessed over an Alibaba Cloud private network, set the value in the `region` format, for example, cn-hangzhou.
`${your_aliyun_user_id}`	The ID of your Alibaba Cloud account. For more information, see Obtain the account ID.
`${your_machine_group_user_defined_id}`	The custom identifier of your machine group. The identifier must be unique in the region where your project resides. For more information, see Create a custom ID-based machine group.

Configure the mount path of logs.
```
volumeMounts:
- name: nginx-log
mountPath: /var/log/nginx
```
- The Logtail container and the application container must share the same directory.
- We recommend that you mount an emptyDir volume on the containers.
Configure the waiting period for the Logtail container after the collection process is stopped.
In most cases, the waiting period is 10 seconds. This value indicates that the Logtail container waits for 10 seconds after it receives a stop command, and then exists. This setting prevents incomplete data collection.
```
command:        
- sh        
- -c        
- /usr/local/ilogtail/run_logtail.sh 10
```

Step 2: Create a Logtail configuration

After you install Sidecar, you can configure an AliyunLogConfig CRD to create a Logtail configuration. The following script shows how to configure a CRD. To delete a Logtail configuration, you must delete the corresponding CRD.

apiVersion: log.alibabacloud.com/v1alpha1      ## The default value. You do not need to modify this parameter. 
kind: AliyunLogConfig                          ## The default value. You do not need to modify this parameter. 
metadata:
  name: simple-stdout-example                  ## The resource name. The name must be unique in the cluster. 
spec:
  project: k8s-my-project                      ## The name of the project. If you do not specify this parameter, logs are sent to the project that is configured when you install Log Service components on the cluster. 
  logstore: k8s-stdout                         ## The name of the Logstore. A Logstore is automatically created if the specified Logstore does not exist. 
  machineGroups:                               ## The name of the machine group. This machine group must be the same as the specified ${your_machine_group_user_defined_id} in your Sidecar. This machine group is used to associate Sidecar and the CRD. 
  - nginx-log-sidecar
  shardCount: 2                                ## Optional. The number of shards. Valid values: 1 to 10. Default value: 2. 
  lifeCycle: 90                                ## Optional. The retention period for which log data is stored in the Logstore. Unit: days. Valid values: 1 to 7300. Default value: 90. The value 7300 indicates that log data is permanently stored in the Logstore. 
  logtailConfig:                               ## The details of the Logtail configuration. For more information, see Logtail configuration files. 
    inputType: file                            ## The type of the data source. Valid values: file and plugin. Container stdout and stderr logs are not supported. 
    configName: simple-stdout-example          ## The name of the Logtail configuration. This name must be the same as the resource name that is specified by the metadata.name field. 
    inputDetail:                               ## The detailed settings of the Logtail configuration. For more information, see Configuration examples. 
    ...

After you create a Logtail configuration, the Logtail container automatically collects and uploads logs to Log Service based on the Logtail confihuration. You can log on to the Log Service console to view the logs.

Notice

The value of the configName field in the CRD must be unique in the specified Log Service Logstore. If multiple CRDs are associated with the same Logtail configuration, the Logtail configuration is affected when you delete or modify an associated CRD. In this case, the status of the other CRDs that are associated with the Logtail configuration is inconsistent with the status of the server.
In Sidecar mode, only text logs can be collected. You must set the dockerFile parameter to false.

Configuration examples

The following example shows how to use a CRD to collect logs in Sidecar mode from a self-managed Kubernetes cluster in a data center. The project that is created in Log Service for log collection resides in the China (Hangzhou) region. Logs are collected over the Internet. The volume that is mounted on the /var/log/nginx directory of the nginx-log-demo and Logtail containers is an emptyDir volume named nginx-log.

Install Sidecar.

apiVersion: batch/v1
kind: Job
metadata:
  name: nginx-log-sidecar-demo
  namespace: default
spec:
  template:
    metadata:
      name: nginx-log-sidecar-demo
    spec:
      restartPolicy: Never
      containers:
      - name: nginx-log-demo
        image: registry.cn-hangzhou.aliyuncs.com/log-service/docker-log-test:latest
        command: ["/bin/mock_log"]
        args: ["--log-type=nginx", "--stdout=false", "--stderr=true", "--path=/var/log/nginx/access.log", "--total-count=1000000000", "--logs-per-sec=100"]
        volumeMounts:
        - name: nginx-log
          mountPath: /var/log/nginx
      ##### logtail sidecar container
      - name: logtail
        # more info: https://cr.console.aliyun.com/repository/cn-hangzhou/log-service/logtail/detail
        # this images is released for every region
        image: registry.cn-hangzhou.aliyuncs.com/log-service/logtail:latest
        # when recevie sigterm, logtail will delay 10 seconds and then stop
        command:
        - sh
        - -c
        - /usr/local/ilogtail/run_logtail.sh 10
        livenessProbe:
          exec:
            command:
            - /etc/init.d/ilogtaild
            - status
          initialDelaySeconds: 30
          periodSeconds: 30
        env:
          ##### base config
          # user id
          - name: "ALIYUN_LOGTAIL_USER_ID"
            value: "xxxxxxxxxx"
          # user defined id
          - name: "ALIYUN_LOGTAIL_USER_DEFINED_ID"
            value: "nginx-log-sidecar"
          # config file path in logtail's container
          - name: "ALIYUN_LOGTAIL_CONFIG"
            value: "/etc/ilogtail/conf/cn-hangzhou-internet/ilogtail_config.json"
          ##### env tags config
          - name: "ALIYUN_LOG_ENV_TAGS"
            value: "_pod_name_|_pod_ip_|_namespace_|_node_name_|_node_ip_"
          - name: "_pod_name_"
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: "_pod_ip_"
            valueFrom:
              fieldRef:
                fieldPath: status.podIP
          - name: "_namespace_"
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: "_node_name_"
            valueFrom:
              fieldRef:
                fieldPath: spec.nodeName
          - name: "_node_ip_"
            valueFrom:
              fieldRef:
                fieldPath: status.hostIP
        volumeMounts:
        - name: nginx-log
          mountPath: /var/log/nginx
      ##### share this volume
      volumes:
      - name: nginx-log
        emptyDir: {}

Create a Logtail configuration.

Set the log path of access logs to /var/log/nginx/access.log and the destination Logstore to nginx-access.

Note In Sidecar mode, only text logs can be collected. You must set the dockerFile parameter to false.

apiVersion: log.alibabacloud.com/v1alpha1
kind: AliyunLogConfig
metadata:
  # The name of the Logtail configuration. This name must be unique in your Kubernetes cluster. 
  name: nginx-log-access-example
spec:
  # The name of the project. 
  project: k8s-nginx-sidecar-demo
  # The name of the Logstore. 
  logstore: nginx-access
  # The machine group to which the Logtail configuration is applied. This machine group must be the same as the value of the [ALIYUN_LOGTAIL_USER_DEFINED_ID] field in your Sidecar. 
  machineGroups:
  - nginx-log-sidecar
  ## The details of the Logtail configuration. 
  logtailConfig:
    # The type of the data source. Set the value to file. 
    inputType: file
    # The name of the Logtail configuration. This name must be the same as the resource name specified by the metadata.name field. 
    configName: nginx-log-access-example
    inputDetail:
      # Set logType to common_reg_log. 
      logType: common_reg_log
      # Specify the log path. 
      logPath: /var/log/nginx
      # Specify the names of log files from which you want to collect data. You can include wildcard characters in the field value, for example, log_*.log. 
      filePattern: access.log
      # Set the dockerFile parameter to false in Sidecar mode. 
      dockerFile: false
      # Use a regular expression to match the start part in the first line of a log. For single-line logs, set the value to .*. 
      logBeginRegex: '.*'
      # Use a regular expression to parse logs. 
      regex: '(\S+)\s(\S+)\s\S+\s\S+\s"(\S+)\s(\S+)\s+([^"]+)"\s+(\S+)\s(\S+)\s(\d+)\s(\d+)\s(\S+)\s"([^"]+)"\s.*'
      # The extracted keys. 
      key : ["time", "ip", "method", "url", "protocol", "latency", "payload", "status", "response-size",user-agent"]
# config for error log

Set the log path of error logs to /var/log/nginx/error.log and the destination Logstore to nginx-error.

# config for error log
apiVersion: log.alibabacloud.com/v1alpha1
kind: AliyunLogConfig
metadata:
  # your config name, must be unique in you k8s cluster
  name: nginx-log-error-example
spec:
  # The name of the project.
  project: k8s-nginx-sidecar-demo
  # The name of the Logstore.
  logstore: nginx-error
  # The machine group to which the Logtail configuration is applied. This machine group must be the same as the value of the [ALIYUN_LOGTAIL_USER_DEFINED_ID] field in your Sidecar. 
  machineGroups:
  - nginx-log-sidecar
  ## The details of the Logtail configuration. 
  logtailConfig:
    # The type of the data source. Set the value to file. 
    inputType: file
    # The name of the Logtail configuration. This name must be the same as the resource name specified by the metadata.name field. 
    configName: nginx-log-error-example
    inputDetail:
      # Set logType to common_reg_log. 
      logType: common_reg_log
      # Specify the log path. 
      logPath: /var/log/nginx
      # Specify the names of log files from which you want to collect data. You can include wildcard characters in the field value, for example, log_*.log. 
      filePattern: error.log
      # Set the dockerFile parameter to false in Sidecar mode. 
      dockerFile: false